#launchpad-dev 2009-08-10
 * thumper frowns
<thumper> tests for db-devel are failing
<thumper> but it is just two merges from trunk
<thumper> and the tests pass on devel
 * thumper relocates
<meaton2veggies> hi guys, how can you control the email address that launchpad sends mail too? is it in lp conf?
<wgrant> thumper: Is that why my latest hasn't been merged yet? It's not the one that has broken the tests, I hope...
<mars> argh, just cc'd mwh, and remembered he's not around for a while :)
<jtv> spm: what's an arvo?
<spm> jtv: slang for afternoon.
<jtv> spm: ah.  In any case, I think that script is just naturally non-noisy.
<spm> jtv: ie "now" :-)
<spm> right
<Ursinha> hello people
<spm> jtv: a non-noisy script is a good script - in my professional opinion. :-)
<spm> Ursinha: errr.... were you including me in "people" - cause I kinda object to that classification. sysadmin donchaknow.
<jtv> spm: absolutely, which is why I made it send even the remaining warnings to the uploader instead of the log.  But it would be nice to know that something is really happening.  :)
<Ursinha> spm, lol
<jtv> hi Ursinha, see you've insulted someone already.  :)
<spm> jtv: we can always disable "-q" ?
<Ursinha> LOL
<Ursinha> see, I just said "hello people"
<jtv> Ursinha: no seriously, I admire that
<Ursinha> hahahaha
<spm> jtv: see! you did it too! "someone". Sigh. I'm so misunderstood.
<spm> ROFL.
<jtv> spm: no, thats not what I meant.
<jtv> spm: I totally understand how you feel about this.
<jtv> spm: it's just that I don't give a smeg.
<Ursinha> omg
<Ursinha> lol!
 * spm literally rolls out of the chair laughing
<jtv> spm: but yes, sure, disable the -q
<spm> jtv: oki
<jtv> wow, look at that guy typing while rolling on the floor... *that* is a good sysadmin
<jtv> (apart from the fact that he's rolling on the floor for any reason except cable management)
<Ursinha> HAHAHAHAHAHA
 * wgrant quickly breaks Launchpad while spm is incapacitated.
<spm> wireless keyboards FTW! surgically attached to my arms.
<spm> wgrant: no need. it can do it all on it's own. :-)
<wgrant> spm: Good to know.
<wgrant> Although it isn't breaking too much these days.
<jtv> wgrant: please file a bug for that... means the cron job is broken
<wgrant> jtv: Is this the one that likes 5 hour transactions?
<jtv> wgrant: wow, that took a surprisingly serious turn...
<wgrant> Ah.
 * wgrant fails.
<jtv> wgrant: 5-hour transactions happen to be a personally sensitive subject.  :)
<jtv> spm: also, would you have a moment(*) to service my staging testing request?
<jtv> (*) for a fairly large value of a moment
<spm> jtv: sure. just give me 5, just mid something atm.
 * jtv hands spm 5
<spm> jtv: ok, we're seeing INFO stuff in that log again.
<jtv> spm: i.e. the script starting and stopping?
<spm> jtv: yup
<jtv> spm: hope that can go somewhere where nagios sees it but we don't have to unless there's anything serious?
<spm> jtv: we use scriptactivity, not nagios for that style of thing. if you need it checked ultra frequently so it's a critical system fail if it doesn't work for <10-15 minutes? we can nagios. ???
<jtv> spm: you make it sound like overkill
<spm> jtv: no not really. it's just degress of prioirty
<jtv> spm: if this script fails, I'd say it's not critical unless it lasts for several hours at least
<jtv> meanwhile, some runs may legitimately take more than 15 minutes.
<spm> jtv: right. and that's where scriptactivity will catch it
<jtv> perfect
<jtv> spm: I see you head several swap alerts for appservers...  do you think that was incidental?  If not, we may have to tweak ORM cache sizes.
<jtv> s/head/had/
<jtv> spm: meanwhile, about those staging script runs...  :-)
<spm> jtv: sorry - that 5 turned ino 45 and then some.
<jtv> spm: call it a round 10Ã?
<spm> jtv: I need to afk in ~ 25 for the school run; but we can progress now.
<jtv> spm: most of this should be noninteractive from your pov
<spm> even better!
<jtv> spm: just run, say, 5 instances of the export script and let me know; then kick off the migration and 5 more exports after that.
<spm> jtv: righto. this is per your email late last week?
<jtv> spm: the rest we can figure out after, assuming staging doesn't restore in the meantime
<jtv> spm: righto indeed
<spm> 'k
<spm> jtv: stage 1 done
<jtv> spm: ack
 * thumper is back
<thumper> I know the answer to the failing test
<thumper> just incase anyone cares
* Ursinha changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 2 of 2.2.8 | https://dev.launchpad.net/ |  Please use #launchpad for support. | https://launchpad.net/~launchpad-dev | Get it: https://dev.launchpad.net/Getting | http://people.canonical.com/~herb/ | http://paste.ubuntu.com/
<Ursinha> what was that thumper?
<thumper> what was what?
<thumper> the fix? or the problem?
<Ursinha> the answer to the failing test
<thumper> I'm not entirely sure how I'm going to fix it yet
<thumper> but the problem relates to security proxies and the object factory
<thumper> now I know the fix \o/
<thumper> nope that's not going to work...
<thumper> ah ha
<thumper> this one will
 * thumper heads for dinner, probably back later
<noodles775> Morning.
<adeuring> good morning
<poolie1> noodles775: do you think it's too crazy to use the same field for search and file-bug?
<poolie1> i kind of like the idea
<noodles775> poolie1: what' the context/use-case?
<noodles775> s/what'/what's
<noodles775> Ah, I see...
<noodles775> Yeah, it would certainly remove an extra click/page-load.
<poolie1> it's possibly overloading it too much
<poolie1> i was thinking "why not just have the file bug thing there "
<noodles775> The "by importance' filtering would be a bit confusing.
<poolie1> and then, really ,with the dupe-finder, the filebug thing really is just like a search field
<noodles775> exactly.
<noodles775> Another thought, if you *just* had a "Search or report a bug" button (somehow... needs thought),
<noodles775> the result being a list of relevant bugs or the option to create your new one.
<noodles775> Nah, that wouldn't work... very different terms entered into the field.
<noodles775> BjornT would be a much better person to think about the above ^^ :)
<BjornT> noodles775, poolie1: i think overloading it might be a bit confusing, hard to get it right so people would understand it. being able to enter the search terms without an extra page load is a good idea. we could you an overlay, or something like that, which pops up when you click on "Report a bug"
<BjornT> (an overlay which would post to +filebug and take you there)
<poolie1> i find about half my interactions are "find an existing bug" and half are "if this isn't a dupe, file it"
<poolie1> if it's an existing bug i usually find it from browser history or emal
<poolie1> email*
<gmb> Morning folks.
<noodles775> hey gmb :)
<gmb> noodles775: Morning! Anything (funny|terrifying|annoying) happen whilst I was away?
<noodles775> gmb: Lots of small things (some great contributions), and there was just some discussion about bugs UI earlier (just 0.5hr ago) you might want to check the log) :)
<noodles775> But nothing terrifying or annoying :)
<gmb> noodles775: Will do. Once I've got through this pile of emails.
<noodles775> gmb: yeah, I'm just finishing up going through the backlog... :)
<gmb> Unfortunately I set up an over-simplistic filter and have not got loads of stuff I normally filter straight to the bin in my "To read" folder
<gmb> s/not/now
<noodles775> You can still update and apply your filters to your inbox though can't you? (both gmail and thunderbird to this iirc)
<gmb> noodles775: Yes. I'm now trying to figure out what I should be filtering out, though :)
<gmb> noodles775: Also, gmail doesn't offer any X-header filtering, which is a real PITA.
<noodles775> Yeah.
<mrevell> Morning
<jtv> hi mrevell!
<mrevell> hey jtq
<mrevell> or jtv
<mrevell> I like that guy too
<jtv> mrevell: I just had it pointed out to me that my blog post about exporting to branches wasn't categorized.
<mrevell> jtv: That's easily fixed. I can go in and do that.
<jtv> mrevell: cool, thanks.
<mrevell> jtv: fixed :)
<jtv> mrevell: thanks
<BjornT> anyone knows why edge hasn't been updated during the weekend?
<henninge> I am wondering why I get (not for the fist time) Ubuntu support questions via the "Contact any user" feature.
<wgrant> gmb: Are you the master of bug migrations?
<gmb> wgrant: Yes, more-or-less.
<wgrant> gmb: Is there anything existing to import a project's bugs from Google Code?
<gmb> wgrant: We've certainly never done a migration from Google Code before, so no, not as far as I know.
<wgrant> gmb: Really? Huh.
<gmb> Unless it was before my time (jamesh used to handle imports)
<deryck> Morning, all.
<danilos> jtv: call
<jtv> danilos: lost you
<danilos> jtv: no, you lost yourself :)
<danilos> jtv: are you around?
<barry> good morning fellow 'padders!
<mrevell> jtv: are you still around, sir?
<barry> bac, sinzui, salgado are we still doing standups? :)
<sinzui> barry: We now have to stand on our heads balancing a bowl of custard. otherwise, nothing has changed.
 * barry makes some custard
<barry> sinzui: sorry, all my bowls are still packed away.  can i use a paper cup?
 * gmb misread the word bowls there, thought there was an 'e' in it.
<barry> ew
<barry> losas: i'm am at your disposal for bug 325962
<mup> Bug #325962: lp-mailman startup is blocking on a pid file in the wrong directory <mailing-lists> <Launchpad Registry:Triaged by barry> <https://launchpad.net/bugs/325962>
<mrevell> cprov: Also, wrt signing PPA keys -- I think we should add a note to the existing pop-up help. Happy with that?
<cprov> mrevell: yes, absolutely
<beuno> noodles775, mars, flacoste, rockstar, intellectronica, jtv, ajax call in 10?
<cprov> mrevell: signing the PPA signing-key doesn't add any trust and in some case cause confusion.
<mrevell> cprov: I'll prepare something
<noodles775> beuno: yep.
<intellectronica> beuno: yes. thanks for the reminder
<flacoste> beuno: i'm sprinting with IS/ISD this week
<danilos> salgado: ping
<salgado> danilos, pong
<danilos> salgado: I am looking at our _getPersonAndEmail code in lib/lp/translations/utilities/translation_import.py, and I am sure you can help me with something
<danilos> salgado: that method calls person_set.getByEmail, and if it doesn't exist calls person_set.createPersonAndEmail
<danilos> salgado: if there's already an emailaddress registered but no person record, it seems to fail spectacularly (or not so spectacularly) with a traceback like http://launchpadlibrarian.net/30100943/nEHny0WhPU3DP4opnCiFlw7QaBo.txt
<danilos> salgado: so, I am looking for a suggestion on how to proceed here? what's the correct thing to do?
<danilos> salgado: and this should give you a clue why I am talking to you: https://pastebin.canonical.com/20963/ :)
<salgado> danilos, to be frank, I'm not sure what to do in cases like this, as simply creating a new Person entry and associating it with the existing account/email will cause the newly created person to show up as a LP user event though the user never registered on LP
<salgado> danilos, that email is associated to an account and not to a person because it's an SSO account
<danilos> salgado: right, I understand what's going on, but I am not so sure what's the right way forward
<salgado> bug 408528 was when I first heard about this problem. https://bugs.edge.launchpad.net/launchpad-foundations/+bug/408528/comments/2 has my reasoning
<danilos> salgado: we've got a lot of person references that already work like this (i.e. translationmessage.submitter is initialized to this person)
<mup> Bug #408528: lucene2 synced from Debian, built fine but failed to upload <Launchpad Foundations:New> <https://launchpad.net/bugs/408528>
<mup> Bug #408528: lucene2 synced from Debian, built fine but failed to upload <Launchpad Foundations:New> <https://launchpad.net/bugs/408528>
<salgado> danilos, right, but since the person has no account associated, they don't show up as LP users -- they show up as placeholder profiles
<danilos> salgado: right, so I do want a person record, but not account record, is that right?
<danilos> salgado: and if it is, what APIs should we use to create it?
<salgado> yes, but that's a constraint we're unnecessarily imposing on ourselves
<salgado> there's no API for that because you need an email address to create a Person entry
<danilos> salgado: right, so would the right thing to do be to change IPersonSet.getByEmail to return a person record even if no account exists, or to change createPersonAndEmail to reuse existing email address, or something entirely different?
<salgado> the correct thing to do here is to create the Person entry we need and associated it with the email/account
<salgado> but we need to somehow flag that new entry as a placeholder profile, so that they don't show up as LP users in the web UI
<danilos> salgado: ah, right, so that's the missing bit right now
<danilos> salgado: is that something we can use creation_rationale for?
<salgado> and we need to fix callsites that do "if IPS.getByEmail(email) is None: IPS.createPersonAndEmail(...)"
<danilos> salgado: right
<danilos> salgado: so, it seems we can't use creation_rationale for that, and we'd need a new db field
<salgado> danilos, a new db field would be best, but it might be possible to abuse creation_rationale for the time being
<salgado> it'd involve some considerable amount of work, though
<danilos> salgado: only if we go back to using binary masks it seems to be, or we'd be losing information; i.e. in this case, I'd like creation_rationale to be POFILEIMPORT | NOTYETUSINGLAUNCHPAD, where NOTYETUSINGLAUNCHPAD > whatever we are using right now (0x100 seems it would do, but something larger would be even better)
<beuno> salgado, hi. How easy would it be to show number of product downloads on the homepage?
<danilos> salgado: right, so any suggestions on how I can fix this bug in a simpler way?
<salgado> danilos, you do need a Person entry, right?
<salgado> beuno, the product's +index page or LP's home page?
<danilos> salgado: right
<beuno> salgado, both?   :)
<salgado> danilos, use a new creation rationale (POFILEIMPORT_NOTYETUSINGLAUNCHPAD) and let the newly created entry show up as a LP user
<salgado> when we fix the real problem we can identify the entries that were created this way and fix them
<danilos> salgado: sure, sounds good... should I strive to make it extractable using binary logical operations, or just make it a next entry in the sequence?
<salgado> danilos, what was the last msg you got from me?
<salgado> beuno, it would be easy to write and cheap to compute for product/+index, but it seems to be unacceptably slow for the home page
<salgado> even the latter might be cheap on production, but would have to check with stub first
<beuno> salgado, cool, I'll file bugs. Thanks
<kfogel> jml: ping (quick q, if you're online)
<kfogel> BjornT: do you know the merging-relationship between devel/db-devel/stable/db-stable ?  I'm writing a wiki page explaining all, but I'm not sure what gets auto-merged where when.
<kfogel> hmm, okay, anyone ^^   :-)
 * kfogel casts about for Someone Who Knows
<bigjools-afk> kfogel: IIRC abentley and/or gary knocked up a web page somewhere about that
<kfogel> bigjools-afk: on the dev wiki?
<bigjools-afk> I don't remember, sorry :(
<bigjools-afk> I just remember that It Was Done
<gary_poster> kfogel: yeah, I wrote one, abentley supplied a nice diagram.  on internal wiki.  looking
<kfogel> bigjools: thanks.  I searched in the dev wiki and didn't find it.
<kfogel> gary_poster: AH!  I searched there too, but no luck.  (search for "devel", "db-devel", "stable", "db-stable" simultaneously gets no hits, but then search is always broken there.)
<gary_poster> kfogel: more diagrams than those at which you might be able to shake a stick: https://wiki.canonical.com/Launchpad/Experiments/DBDevel
<kfogel> gary_poster: OMG,that's perfect.  Thank you.  I will translate to dev wiki.
<gary_poster> cool
 * gmb EoDs.
<intellectronica> rockstar: when running the full test suite on EC2, i got some errors that look code related. the branch doesn't really touch anything related to them. see https://pastebin.canonical.com/20973/ any ideas?
<intellectronica> jml: maybe you know? ^^^^^
<intellectronica> this is a branch of db-devel, b.t.w
<jml> intellectronica, no idea, I'm afraid.
<rockstar> intellectronica, that looks like a test abentley added.  Is it failing in trunk?
<intellectronica> rockstar: it is failing in db-devel
<rockstar> intellectronica, disable it and file a bug.  I'll take a look at it.
<intellectronica> rockstar: k
<rockstar> intellectronica, thanks
 * rockstar goes back to ignoring IRC
<maxb> How long should I wait after submitting a contributor agreement before chasing if I haven't heard an acknowledgement?
<kfogel> maxb: it shouldn't take long.  To whom did you send it?
<maxb> contributor-agreement@ and kiko@
<kfogel> maxb: can you resend to me and contributor-agreement@ ?  I'll get it in.
<kfogel> maxb: I think kiko's just having a busy week.
<kfogel> Not that I'm not having a busy week, but (I suspect) not like kiko's.
<kfogel> :-)
<maxb> I only sent it late last wednesday - I can wait a bit longer if it's likely to be in the pipeline
<maxb> I still need to get around to sorting myself a non-karmic chroot to test my changes in
<kfogel> maxb: well, actually, let me check internally, one sec
<kfogel> maxb: is your middle name Oliver?
<maxb> yes
<kfogel> maxb: then we have it on file :-)
<maxb> no one told me :-)
<kfogel> yeah, I'm going to follow up about that internally
<kfogel> maxb: ^^
<maxb> But that's fine, now I have no excuse not to get on submitting some actual branches :-)
<intellectronica> andrea-bs: hi, tests run cleanly and i'm now landing the branch on your behalf. thanks again for working on this (and for doing such excellent work with so little guidance)
<andrea-bs> intellectronica, thank you very much for your help!
<beuno> bigjools, is your mockup based on an actual branch, or gimped??
<cprov> beuno: 50/50, AFAIK
<beuno> cprov, ok, because I think we're entering a phase where visual changes are needed mora than layout changes
<cprov> beuno: btw, can you take a look at https://dev.launchpad.net/SoyuzBuildIndexPage as well?
 * beuno looks
<cprov> beuno: yes, I realized this on my changes as well, at this point it's not any harder to tweak the templates directly and the result is way more useful than mockups
<beuno> cprov, that mockup I *really* like
<cprov> beuno: the build-details table was required by buildd-admins, making it super-ugly is my fault.
<cprov> beuno: note that the portlets on the 2nd line of the grid may grow vertically according to the number of resulting binaries.
<cprov> beuno: so, having the details table doesn't affect it badly, since the 1st line of the grid will have an almost fixed height.
<beuno> cprov, yeah, agreed
<cprov> beuno: it just need to be easier to the eyes, the current <table class="listing"><tr class="shaded/white"> didn't work, right ?
<cprov> beuno: see an example of a huge list of binaries -> https://dogfood.launchpad.net/ubuntu/+source/linux/2.6.28-11.36/+build/913196
<beuno> cprov, yes, this table is much nicer
<cprov> beuno: cool, the idea is to split context and status of a build
<beuno> cprov, sounds like a great plan
<beuno> this page still looks too messy: http://people.canonical.com/~ed/dsp_mockup2.png
<cprov> beuno: as in general TMI or the layout itself ?
<beuno> cprov, layout
<beuno> 100% layout
<cprov> beuno: I think a bugtask-like table for the published version info would be nicer because of its fixed size and colored lines.
<beuno> cprov, agreed
<beuno> which is kind of why I want to get my hands on the branch
<cprov> beuno: but I also see that it's not very different than what we have and people dislike :(
<beuno> cprov, I don't really understand all the issues behind this
<beuno> maybe we should schedule a call and some time to pair on this?
<cprov> beuno: sure, now ?
<beuno> cprov, can't, need to un-swamp myself. How's Wed?    :)
<cprov> beuno: works fine.
<beuno> cprov, 15UTC?
<cprov> beuno: burning my lunch-time ? :)
<cprov> beuno: 1h later or earlier ?
<beuno> cprov, ok, earlier
<beuno> 14UTC
<beuno> 10am my time
<cprov> beuno: deal.
 * beuno calenders it
<kfogel> andrea-bs: https://dev.launchpad.net/Trunk  is still in progress, but please see if it answers your questions.
 * andrea-bs looks
<andrea-bs> kfogel, The images really help. I think that the text is a bit hard to understand, especially because: 1. when I read "stable" I think to launchpad.net, not edge.launchpad.net; and 2. what exactly is testfix?
<andrea-bs> kfogel, but yes, it answers my questions
<kfogel> andrea-bs: yeah, a lot of work to do there.  The internal wiki page I'm translating from has a wealth of information, but was slightly out-of-date and assumes certain prior knowledge.
<bigjools> beuno, it started off as a template but most of it's gimped now.  I suck at gimp. can't you tell? :)
<Lantash1> @danilos: It's me, the guy attempting to fix #392154. Thanks alot for introducing me to the test environment with your verbose explanation. However, I couldn't complete the testcase because pofile.contributors seems to be empty for some reason: http://pastie.org/578867
<mup> Bug #392154: translator-credits show two contributor lists <Launchpad Translations:Triaged> <https://launchpad.net/bugs/392154>
<danilos> Lantash1: hi
<barry> kfogel: ping
<kfogel> barry: pong-a-long-long
<barry> kfogel: hi!  i'm looking at dev.lp.net/Trunk
<danilos> Lantash1: it's possible we need to do a pofile.sync() before the contributor shows up (a list of contributors is updated with a database trigger, i.e. behind the scenes)
<barry> i'm not sure i understand what you're trying to say by "corresponds to"
<Lantash1> danilos: Thanks. I'll give it a try.
<danilos> Lantash1: cool, let me know how it goes
<Lantash1> danilos: Don't seem to be allowed to do that: "ForbiddenAttribute: ('sync', <POFile...>)"
<barry> kfogel: i mean, i think i know what the intent is :) but the wording is awkward
<kfogel> barry: I'm planning to work on the page more today, but just go ahead and fix/edit/tweak/rewrite/whatever anything there!  It's a VERY rough draft; based on the internal wiki page gary_poster pointed me to (see the top, there's a link).
<barry> kfogel: cool, i didn't want to step on any toes!  let me see if i can tweak it
<kfogel> barry: no, the toes are already stepped on; all you can do is set & mend them now.
<kfogel> (not to imply that you did any stepping, barry -- I mean these toes come pre-stepped!)
<barry> kfogel: i'm better at digging out ingrown toenails :)
<barry> kfogel: thanks!
<kfogel> barry: Ok, I'm not sure where this metaphor is going, but I think this is my stop.  Got to get off.  Bye!  Have fun stormin' the castle!
<barry> kfogel: tweaked.  please let me know if i made it worse
<kfogel> barry: oh, that is *much* better, thank you.
<kfogel> andrea-bs: see above
<kfogel> barry: any further improvements or clarifications you think that page could use, please just make 'em.
<kfogel> barry: if I'm editing, you'll get a warning, but I'm not right now
<kfogel> (and won't be for a few hours)
<andrea-bs> barry, kfogel, it's much better now (but of course the whole page can be improved)
<barry> kfogel: cool.  i think i'll leave it for now.  i'm mostly in email catchup mode so trying to just hit the low hanging fruit as i go.
<barry> andrea-bs, kfogel i was thinking that the picture might want to come earlier, but i won't touch that for now
<barry> kfogel: and i might be pinging you again in a bit when i catch up on the "contributions" thread
<kfogel> andrea-bs: I agree with barry about the pictures coming earlier -- if you want to Just Do It, go for it.
<andrea-bs> barry, Absolutely yes! To understand the text I had to look at the picture :)
<barry> andrea-bs: exactly!  i've known this stuff for months now, but the picture always reminds me so much quicker than the words :)
<andrea-bs> kfogel, I'm a bit tired now (it's late here), so I think I will move it tomorrow if nobody else does it. Sorry!
<danilos> Lantash1: sorry for the delay, busy with some stuff
<danilos> Lantash1: you can avoid that in tests by using removeSecurityProxy(pofile).sync() (from zope.security.proxy import removeSecurityProxy)
<Lantash1> danilos: I'm sorry to disappoint you, but pofile.contributors is empty even if a sync is performed on the pofile just before accessing it.
<kfogel> andrea-bs: I'll do it, np.
<danilos> Lantash1: ok, perhaps a full transaction.commit() is necessary, let me try it out
<salgado> sinzui, ping.  I'm having trouble creating a navigationmenu to use in the +index page of project groups
<sinzui> salgado: for Change details and Administer?
<salgado> sinzui, yes, these and 'Subscribe to bugmail' (which would require work to move to another facet)
<danilos> Lantash1: ah right, you need a sequence=1 when creating potmsgset
<sinzui> salgado: I had a similar thought
<sinzui> salgado: milestones and series have the link in their index page
<sinzui> salgado: We should be sure what ever we do, pillar, series, and milestone do it the same way
<danilos> Lantash1: so, this worked for me
<salgado> sinzui, when I created the new menu (https://pastebin.canonical.com/20984/), the template OOPSes when attempting to access context/menu:overview
<danilos> Lantash1: this is because with our recent message sharing work (we still tend to forget some bits due to being used to different behaviour), potmsgset doesn't have to be attached to a template even if it's added to it
<salgado> sinzui, https://pastebin.canonical.com/20985/ is the OOPS
<danilos> Lantash1: so, adding a sequence parameter to makePOTMsgSet call attaches it explicitely
<sinzui> salgado: I have seen that when the code the make the link is bad
<danilos> Lantash1: to be honest, it's actually a crappy factory method, but just because nobody had time to make it better yet (i.e. it should automatically calculate a suitable sequence number and use that)
<Lantash1> danilos: Adding the sequence argument solved the problem. Thanks for your help! I saw that other makePOTMsgSet factory calls use the sequence argument but didn't know what it is for, so I removed it.
<sinzui> salgado: you create a nav menu for a view, not the context
<danilos> Lantash1: right, it's useful to have the possibility to create it unattached (especially for testing), but in many tests you do want it linked to a potemplate
<Lantash1> danilos: I'll try to write as many useful and reasonable assertions as possible in order to cover the bug and get back to you.
<sinzui> salgado: try
<sinzui> add implements(IProjectActionMenu)  to ProjectView
<sinzui> and call <tal:menu replace="structure view/@@+global-actions" />
<salgado> the first line I had already
<salgado> just tried using view/@@+global-actions instead of context/...
<salgado> didn't work
<salgado> erm
 * salgado uncomments the implements() line
<danilos> Lantash1: ok, cool... it's already quite late here, so I'll be out soon, but you can freely link the branch to a bug and comment on the bug, or you can even ask for a review using merge proposal if you have what you believe is code we can discuss; don't feel blocked even if it's far from complete, it's best to discuss it early on, and we can help whenever needed :)
<danilos> Lantash1: btw, thanks for taking interest in the bug, it's very appreciated :)
<salgado> sinzui, same error, with implements() uncommented and using view instead of context
<sinzui> salgado:  your template is dying on the overview menu, not your @@+global-actions
<danilos> Lantash1: note that the important bit is that you now "import" (i.e. use makeTranslationMessage(is_imported=True) a translation generated with this, because that's the only way to get a translation for credits message in
<salgado> sinzui, right, but that doesn't happen if I comment out the new menu
<sinzui> salgado:  I saw a lot of ambiguous failure in meuns because of tales. There is a problem in the one of the menus such as making a link
<sinzui> salgado: am I right in seeing that you removed edit and administer from one menu, but did not remove them from the list of link
<salgado> oh, crap
<salgado> that's it
<salgado> must be it
<sinzui> salgado: if you want two menus to share a link, use a mixin
<salgado> I don't
<salgado> just forgot to remove
<sinzui> salgado: I am assuming you are showing me the whole diff, so the remove must still happen
<salgado> yes, that's what I forgot and is probably what's breaking the page
<salgado> sinzui, now it works, but it crashes when context/@@+global-actions is not found.  it might be that I'm missing a merge from mainline, though
<salgado> sinzui, thanks!
<sinzui> salgado: your code does not bind the menu to a context
<salgado> I just followed the instructions on https://dev.launchpad.net/VersionThreeDotO/UI/Conversion
<sinzui> salgado: you created a marker interface that we use to bind the menu to a view
<sinzui> salgado: to create one for a context object:
<sinzui> class MyMenu
<sinzui>     usedfor = IObjectType.
<sinzui> same as the navigation menus on person and product
<salgado> right, but if it's only used in one page, why would I bind it to the context instead of the view?
<sinzui> salgado: In the case you need to use an action menu and a related pages menu. I cannot think of a real need for that case. the action menu should be only be used on an index page, and I cannot image a related pages section on an index page too
<sinzui> salgado: We get the context option to create a menu because Navigation menus were designed 18 months to support two levels per the 2.0 UI
<salgado> right
<Lantash1> http://www.youtube.com/watch?v=AYM-_qfytfA&eurl=http%3A%2F%2Fde-bug.de%2Fmedien%2Farchives%2Fwahlspot-dilemma.html&feature=player_embedded
<Lantash1> sry everyone
<Lantash1> didn't meant to post this link here
<rockstar> thumper, when you're awake, children have been to school, etc, ping.
<dobey> i seem to be getting a lot of timeouts lately
<dobey> OOPS-1318EC206 is one of them
<beuno> sinzui, yo
<sinzui> hi beunio
<sinzui> beuno
 * rockstar goes on a short walk
<beuno> sinzui, would you like to have a catch-up call?
<sinzui> beuno: in 15 minutes
<beuno> sinzui, sure. Call me when ready
<sinzui> beuno: I'm calling
<beuno> sinzui, headphones on, waiting eagerly
<beuno> EdwinGrubbs, hi
<EdwinGrubbs> beuno: what's up?
<thumper> rockstar: ping
<thumper> rockstar: I'm home along :)
<thumper> s/along/alone/
 * thumper needs more coffee
<rockstar> thumper, get more coffee, then we can skype.
 * rockstar secretly fears an un-caffeinated thumper
<thumper> rockstar: I have coffee, we can skype now
<beuno> EdwinGrubbs, I've just purchased your time to work a bit on the timeline after you land the team page
<beuno> are you up for it?
<beuno> I can still get a refund
<EdwinGrubbs> beuno: working on the timeline would be fine.
<EdwinGrubbs> sinzui: ping
<sinzui> Hi EdwinGrubbs
<EdwinGrubbs> sinzui: I read your email. Can I use view/+global-actions instead of context/+global-actions or some other menu tied to views, since the +related-pages are supposed to be inline, but I need certain actions only on the edit pages.
<sinzui> EdwinGrubbs: yes. I think we should avoid menus on the context object for now
<EdwinGrubbs> sinzui: won't the menu being registered on the view for +related-pages interfere with registering one for the view on +global-actions? How can I place one menu inline and one menu in the sidebar that both use the view as the context object.
<sinzui> EdwinGrubbs: I am confused. If a page needs two menus then you must define a a nav menu on the context object and a nav menu for the view. I have not seen a reason for any page to need two menus.
<sinzui> EdwinGrubbs: what is your need for two menus on the team page>
<EdwinGrubbs> sinzui: ok, so I will have a context/+global-actions inline and a view/+global-actions in the sidebar?
<sinzui> EdwinGrubbs: You are killiing me
<EdwinGrubbs> sinzui: and menus are killing me.
<sinzui> EdwinGrubbs: The action menu may only go in the sidebar...thus can only be used on pages that have a side bar. we think index pages of object are the only pages that get a sidebar
<sinzui> EdwinGrubbs: what page are you designing?
<EdwinGrubbs> sinzui: sorry, I flipped the pages in my head. My question should have been, do I use view/+global-actions in the sidebar and context/+related-pages inline, which is reversed from your original example?
<sinzui> EdwinGrubbs: yes. That is how they are intended
<sinzui> EdwinGrubbs: are you designing the team page?
<EdwinGrubbs> sinzui: the $team/+edit page.
<sinzui> EdwinGrubbs: Edit pages (form pages) do not have a sidebar, they cannot have an action  menu. You may choose to have a related pages portlet after the form like the product-edit.py page
<sinzui> EdwinGrubbs: If you are concerned about common links (Administer, Change details), then I advise you place them in a mixin class. Use the mixin in every NavigationMenu that need one of those links
 * sinzui looks for example classes
<EdwinGrubbs> sinzui: ok, that is fine. I had complained in another thread about the "Change branding", "Change owner", etc. links being at the bottom of the "Change details" page, where they are easy to miss. That's why I was trying to stick everything in the sidebar, but I'll stop doing that now.
<sinzui> EdwinGrubbs: take a look at lp/registry/browser/product:
<sinzui>     ProductEditLinksMixin:
<sinzui>     ProductEditNavigationMenu(NavigationMenu, ProductEditLinksMixin)
<sinzui>    ProductOverviewMenu(ApplicationMenu, ProductEditLinksMixin):
<sinzui> and the templates/
<sinzui>     product-edit.pt
<sinzui>     product-admin.pt
<sinzui>     
<sinzui>     
<sinzui> EdwinGrubbs: That is a big NO
<sinzui> EdwinGrubbs: the sidebar is a distraction from the single task the user has to perform
<sinzui> The conversion page does say that modifcation pages are main_only
<sinzui> EdwinGrubbs: remember, We expect most users to be changing content inline, so we want the links inline
<EdwinGrubbs> sinzui: Then, I think it is weird not to have easier access to "Change branding", etc. on some other page. If I can put an edit link right next to the big team logo, that would solve that problem, but it might look ugly.
<sinzui> EdwinGrubbs: I do not think you are thinking like a user. How does the user no branding is not on the page until he has scrolled to the bottom? Why should we assume that when the user first sees the page, that something is wrong and he wants to leave?
<sinzui> EdwinGrubbs: The branding issue is special
<beuno> you know nwhat would be super mega awesome for branding?
<beuno> an edit link right under the picture
<sinzui> EdwinGrubbs: Lifeless proposed a solution, but the new design makes it hard. We will solve the problem for all IHasLog in in glorious landing
<EdwinGrubbs> sinzui: The user's thought pattern would be, "I used the change-branding page the other day, and now I want to go back to it. There is no link on the overview page, so I'll click on 'Change details'. Oh look, there are no links in here, maybe they put the branding form fields in the change-details form. Let me look carefully through all the form fields on this page. When I see the form's submit button, I assume that there is no
<EdwinGrubbs> because everything below the form must be part of the footer.
<sinzui> EdwinGrubbs: I think you are wrong. After clicking Change details, The user looks for the branding form, see it is not there, a but a link is present.
<sinzui> EdwinGrubbs: And the correct fix is not to assume something is wrong, but to make something right. We need to solve two things, which an action menu does not...
<sinzui> EdwinGrubbs: beuno: The user needs to see the default art and know he can click near it to set the value. Branding makes no sense in regard to a person or team. No sane user would see Change branding and click on it to set their picture.
<beuno> sinzui, agreed
<beuno> "branding" is the wrong term
<beuno> even for project IMHO
<sinzui> EdwinGrubbs: beuno: lifeless suggested we make the default image a link to set branding. Now it is a link to the pillar/person
<beuno> there's 2 easy ways out of this
<beuno> sinzui, yes, and we could compliment that with a different image saying "upload picture"
<beuno> that just the ownser sees
<beuno> changing your picture *again* is a less common operation
<sinzui> beuno: I am updating the bug now. You win a gold â
<beuno> so you could show the edit link on hover
<beuno> wooooo
 * beuno copies and pastes that star to /home/beuno/chest
<sinzui> beuno: I am noting hover too
<beuno> super\
<sinzui> beuno: I will get this fixed by 3.0
<beuno> sinzui, it will be an epic release
<gary_poster> jml: ping?
#launchpad-dev 2009-08-11
<thumper> sinzui: ping
<thumper> beuno: ping
<beuno> thumper, hi
<thumper> beuno: hi
<thumper> beuno: the new heading that we have for 3.0 is bad for branches
<thumper> beuno: we spent quite a bit of time getting the breadcrumbs working for branches
<beuno> thumper, enlighten me please
<thumper> to show the branch target
<thumper> eg. person, product or source package
<thumper> now it is just showing the person
<thumper> (the first part of the url)
<thumper> it has to do with how we select the relevant context
<thumper> beuno: see BranchHierarchy in lp.code.browser.branch
<beuno> thumper, so you're saying that the context of branches is people, no products?
<thumper> beuno: in lp.dev for the new layout, it is showing the person not the project
<beuno> thumper, right, it should show the project
<thumper> right
<thumper> we agree
<thumper> it is broken right now
<beuno> ok
<beuno> how can I help?
<thumper> who did the work on the heading?
<beuno> Curtis
<thumper> we need to get that context right
<thumper> ok
<thumper> I'll talk to sinzui about it
<thumper> beuno: thanks
<beuno> thumper, you're welcome
<beuno> now, shower and leaving before I become single again
<thumper> beuno: ouch
<thumper> beuno: I'd like a call with you in about 20h if that is good with you
<beuno> thumper, maths escape me, but I think that's doable
<thumper> beuno: think back 4 hours
<thumper> beuno: and add a day :)
<beuno> thumper, I should be available, yes
<beuno> and thanks  :)
<thumper> :)
 * beuno pretends to close his laptop
<meaton2veggies> hi guys, with launchpad and postfix, how is it meant to be setup so that mail can be sent externally say when someone registers etc
<wgrant> meaton2veggies: That sounds dangerous, and you very probably don't want to do it.
<meaton2veggies> wgrant: just wanted to have mail sent properly not working out so far - have a open relay on the internal network struggling with postfix config, but where should I config Launchpad for which smtp host it uses? possible?
<wgrant> meaton2veggies: Why won't local mail do? Are you trying to run your own internal instance?
<meaton2veggies> yes
<meaton2veggies> wgrant: trying to run internal instance
<meaton2veggies> wgrant: not just locally
<wgrant> meaton2veggies: You know you can't run a non-development non-test instance with the images that are provided, right?
<meaton2veggies> wgrant: yeah
<meaton2veggies> wgrant: wanted to have some other devs testing with it and connected to internal mail rather than just on one box
<meaton2veggies> wgrant: but guess I should just use the local mail
<meaton2veggies> wgrant: so if wanted to use for non-dev later whats the license/rules is it documented somewhere?
<wgrant> meaton2veggies: See LICENSE.
<meaton2veggies> got it
<sinzui> thumper: the branding object displayname and tabs are Michaels work. I added the heading slot which is just a <h2> and is meant to be overridden when it does not do the right thing
<wgrant> Ahem.
 * wgrant reports a nasty bug in the new header code.
<wgrant> Odd that Ohloh is having problems importing LP (http://www.ohloh.net/forums/3491/topics/3685). Although that command took 2.5 hours to run, it worked fine here...
<spiv> wgrant: yeah, that does seem odd.  I wonder how easy it would be to encourage them to file bugs about their bzr issues...
<wgrant> spiv: I wonder...
<wgrant> Although my test was against devel, not db-stable, I don't imagine there'd be much difference.
<rockstar> thumper, hey
<thumper> rockstar: hey
<thumper> whazzup?
<rockstar> thumper, just got your email.
<thumper> ah
<rockstar> You've just discovered the most annoying thing about the overlay widget: it's fixed width.
<thumper> why?
<rockstar> nfi
<thumper> arse
<rockstar> So, I hacked around it in the branch subscription stuff.
<rockstar> thumper, https://pastebin.canonical.com/20998/
<thumper> we
<thumper> ew
<rockstar> thumper, I know, but it's better than what you have now.
<thumper> :)
<thumper> we'll see what beuno says
<cprov-zzz> g'night, guys.
<noodles775> Morning
<wgrant> noodles775: Morning. I think it's you that might want to look at bug #411738.
 * noodles775 looks
<noodles775> gar, thanks wgrant
<wgrant> noodles775: Fortunately edge hasn't updated recently.
<noodles775> yep
<gmb> Morning folks.
<wgrant> Morning gmb.
<gmb> Hi wgrant. How's things?
<wgrant> gmb: It's tempting to follow your recent identi.ca trends, complaining about JavaScript...
<gmb> wgrant: Hah. Yes, well. My complaints about JS are *nothing* compared to my complaints about PHP. But I never do any PHP any more, thank goodness.
<wgrant> gmb: This codebase is fortunately now PHPless, however it has some... suboptimal JavaScript.
<gmb> Oh hell yes.
<gmb> Although I sometimes wonder if it's possible to write optimal Javascript.
<gmb> And although we're PHP-less there's still some Perl in the tree, which makes me shudder every time I have to go near it (i.e. Debbugs imports).
<wgrant> jQuery/YUI/ANYTHING makes it significantly more optimal than this stuff.
<gmb> True.
<maxb> Hm. I'm attempting to run tests in python 2.4 to establish a baseline to test my changes against, but I get failures in a pristine devel branch
<noodles775> maxb: what are the failures?
<maxb> http://paste.ubuntu.com/251210/
<wgrant> There was a testfix an hourish ago.
<wgrant> That's at least one of those.
<wgrant> (menus.txt)
<maxb> ah
<wgrant> The two errors are because you're chrooted.
<wgrant> They're fine.
<wgrant> Not sure about the other failures.
<BjornT> xx-bugtracker-remote-bug.txt was also fixed by the testfix commit
<mrevell> Morning!
<wgrant> BjornT: Oops, missed that.
<wgrant> That question in #launchpad reminds me... apart from the hardcodings in LP, is there anything stopping Debian PPAs from working?
<Knut-HB> Hi, I have question concerning accessing launchpad from the www: a team member asked me if it is really necessary to use https or if http could be used instead and if I could forward this question.
<maxb> I've asked this in the past and been told that it's easier to just require https for everything than try to bounce you back and forth during a session depending on whether what you're looking at would be visible to an anonymous user or not.
<wgrant> maxb: It can't possibly do that, because the HTTP pages wouldn't know that you were logged in.
<wgrant> So they could never bounce you back to HTTPS.
<maxb> You could contrive a setup where they did, but it would be oh so complicated and not worth it
<Knut-HB> so it is rather useless to try and change everything from https to http?
<maxb> yes
<Knut-HB> ok thanks
<wgrant> bigjools: Are there any non-technical reasons for the lack of Debian PPAs?
<bigjools> yes, the time and effort involved to set them up
<jtv> danilos: this better?
<danilos> jtv: yeah
<danilos> jtv, henninge_: call time
<jtv> danilos: madness reigns.
<danilos> jtv: https://devpad.canonical.com/~henninge/mockups/translate-page/list-focused.html
 * gmb wonders why people sometimes file questions against launchpad.net/null. What are we doing that makes them think that's the right place to go?
<wgrant> gmb: Ahahahahaha.
<wgrant> gmb: The Answers help page was changed to point to that exactly because people kept filing random questions against the old target.
<wgrant> People are even more braindead than I thought.
<gmb> wgrant: !
<gmb> That's vexing.
<henninge> danilos: I updated the page to have 1/3-2/3 column widths, thanks to yui.
<henninge> jtv: ^
<henninge> But I am not sure it looks better.
<wgrant> https://help.launchpad.net/Answers/AskingForHelp is the guilty page
<jtv> henninge: I think it does actually
<wgrant> Oh, and can somebody please remove NULL from launchpad-project?
<wgrant> It keeps giving me useless bugmail.
<wgrant> Or is there a good reason it's there?
<henninge> jtv: ok, I don't really care that much, so I could go with it.
<jtv> henninge: I think this is a major jump forwards in any case; maybe martin should worry about the pixel-polish
<henninge> jtv: did you know that YUI's cssgrids is deprecated in the beta and will be completely redone for the final release?
<danilos> jtv: eventually, we'll end up worrying about it as well
<danilos> henninge: yaaay!
<jtv> henninge: I did not know that, and I still sort of wish I didn't  :)
<danilos> henninge: I think we should just switch all our servers to use karmic ;)
<henninge> danilos: that is YUI3's beta and final I was talking about
<danilos> henninge: btw, I always loved it how width:100% never really worked on textareas, if that's what you are using there
<henninge> danilos: yes, I noticed. Didn't know that before.
<danilos> henninge: btw, one thing that we should definitely keep: add the copy button between English text and input boxes
<henninge> danilos: np
<jtv> that's another thing: if this fills up with knobs and boxes again, we definitely need the extra space on the "input" side of the form.
<gmb> wgrant: I'm not sure why it's there. Might be worth starting a conversation on launchpad-dev to find out why it is where it is.
 * henninge lunches
<wgrant> gmb: Thanks. I shall fire off an email.
<danilos> jtv: we should try not to let it fill up with knobs and boxes again
<jtv> danilos: you just told henning to add one.  :-P
<jtv> (I get your drift though :)
<danilos> jtv: ;)
<danilos> henninge-lunch: btw, a bit nicer icons, or at least so I think :)
<danilos> henninge-lunch: https://devpad.canonical.com/~danilo/screenshots/henninge+translate.png
<jtv> danilos: I find the menu bar a bit hard to read on that one...  also, wonder what it'll look like with the â horizontal layout?
<danilos> jtv: sshhh
<jtv> danilos: the "package" icon is very... apt.  :)  What does the hand mean?
<danilos> jtv: hand means slapping someone, or sharing :) it'd just "better" than what henninge-lunch had originally, it's far away from perfect
<jtv> danilos: maybe there's something you can rip off in U1?  (Hope we don't have to support Ubuntu themes though :)
<danilos> jtv: I wouldn't worry about it too much right now, we want it to look reasonably good so we can discuss the bits that are missing
<jtv> danilos: sure
<danilos> jtv: I believe the interactions need to be modeled well, and that's the first thing we need to do next
<jtv> danilos: right, "someone needs to review" and such
<danilos> jtv: exactly
<mrevell> http://planet.launchpad.net -- not yet themed and still adding feeds
<gmb> danilo: Can you or another translations person take a look at https://answers.edge.launchpad.net/launchpad/+question/79641? I'm not sure whether he's just asking for a team or for something else (which is a sign that I need to brush up on my Translations knowledge, alas).
<barry> losa ping
<gmb> bigjools: Can you or someone else Soyuzy look at https://answers.edge.launchpad.net/launchpad/+question/79709 please? I don't know if this is user error or a bug.
<wgrant> User error.
<bigjools> gmb: I can answer
<bigjools> pebkac
<gmb> bigjools: Thanks.
<wgrant> Very strange PEBKAC, though.
<henninge> danilos: have a look at this, please: https://devpad.canonical.com/~henninge/mockups/translate-page/list-noradio.html
<henninge> danilos: it shows some more dynamics and also my idea of getting rid of radio buttons (yuk!).
<danilos> henninge: it looks good, but radio buttons must be there
<henninge> :-(
<danilos> henninge: I know you try to push for it, but there's no convincing me
<danilos> henninge: and if you try harder, I'll get mpt to come over :)
<henninge> ok, ok ...
<henninge> no, please, no!
<henninge> ;-)
 * henninge should have checked for mpt's nick in the list first ...
<danilos> henninge: you don't have to trust me, but if you do a user survey where you show that this would work better than radio buttons, I am sure many usability experts will want a chat with you :)
<henninge> can't be bothered  ...
<danilos> henninge: anyway, other than discoverability, you'd also have to implement many features radioboxes already provide, like keyboard navigation and such
<danilos> henninge: believe me, this might look prettier to you (though honestly, it doesn't to me :), but is far less usable
<danilos> henninge: also, just move icons to the right, and you'll see how radio boxes are not ugly
<danilos> henninge: add <label for=...> elements as well, so that clicking the text and icon works well as well
<gmb> bigjools, cprov, noodles775: Could one of you guys field a PPA-siging question in #launchpad please?
<salgado> sinzui, any idea what change in mainline (or your product-page-3.0 branch) would cause this gap at the top of one of my yui-u? (https://devpad.canonical.com/~salgado/project-group.png)
<salgado> that gap appeared after I merged from your branch yesterday
<sinzui> salgado:  using yui classes and porlet on the same div? That is bad.
<sinzui> salgado: https://dev.launchpad.net/VersionThreeDotO/UI/Conversion#Example%20layouts shows that we must separate yui-g and yui-u from our classes. This is a forced separation of YUI layout and launchpad layout
<sinzui> hmm
<sinzui> projectgroups have latest bugs and blueprints. projects do not. I think I should add them this week
<salgado> sinzui, I had removed 'portlet' from the classes after I read that yesterday, but didn't save the file after that.  yesterday was definitely not a good day
<noodles775> salgado: you need to ensure that your div with the class 'portlet' is wrapped inside a div with the yui-u class (ie. not have them on the same div)
<noodles775> salgado: I had the exact same problem.
<noodles775> ah, as already said above...
<sinzui> salgado: noodles775: I discovered early last week that the portlets I marking with yui and launchpad classes were not portable. They broke other pages. That is to say all templates that want to use the questions portlet had to use it in the same king of layout.
<salgado> sinzui, noodles775, do I really want the extra <div class="portlet"> inside the yui-u?
<sinzui> salgado: noodles775: So I separated the too
<sinzui> salgado: You should.
<noodles775> sinzui: yes, it makes sense when thinking about re-using portlets on multiple pages. I hadn't thought about that.
<sinzui> noodles775: clearly I did not think about it too when I sent my draft rules.
<henninge> danilos: Here, includes radio buttons and copy button but no labels (requires id's to be set).
<henninge> https://devpad.canonical.com/~henninge/mockups/translate-page/list.html
<danilos> henninge: looks much, much better
<henninge> danilos: glad to hear.
<henninge> danilos: have you talked to Martin?
<henninge> danilos: I have to go into town now to take care of something.
<henninge> danilos: I remember you wanted to leave early today?
<danilos> henninge: I am about to leave
<henninge> danilos: ok, see you tomorrow!
<danilos> I haven't had a chance to talk to beuno yet, but we'll be hunting him down tomorrow and on Thursday :)
<henninge> cool, that'll be fun. I never went hunting before!
<barry> losa ping
<kfogel> sinzui: wgrant's message of https://lists.launchpad.net/launchpad-dev/msg00329.html -- his suggestion that the 'null' project not be part of Launchpad Suite seems very sane to me.  Any other factors here we should know about, or is this a JFDI?
<sinzui> kfogel: I know that a canonical employee created the project. It is not used the way we intend bugtasks to work.
<sinzui> kfogel: As I am tasked with solving the project-part-of problem, I think we can ask an admin to remove the project because we did not create it.
<sinzui> kfogel: I have demi-god powers. I think I can remove it right now!
<kfogel> sinzui: go for it!  But you mean remove 'null' from 'Launchpad Suite', right?  (Or do you mean remove 'null' entirely?)
<sinzui> correct
<sinzui> kfogel: it worked!
 * sinzui likes being and registry-expert
<kfogel> sinzui: you demi-glazed demi-god, you
<sinzui> :)
<sinzui> kfogel: I presume /null was gifted to the registry-expert team because the real owner did not want bug mail.
<kfogel> sinzui: who knows.  Is anyone going to get bug mail now?
<sinzui> Not the team I am in.
<sinzui> There are only 21 bug in the null project
 * gmb EODs
<mrevell> okay see you tomorrow people
<barry> leonardr: how do you handle circular references in lazr.restful webservice declarations? :(
<leonardr> barry: set the reference to Interface and then go in later and re-set it
<leonardr> there are many examples throughout launchpad + some helper functions. look in interfaces/user.py
<barry> leonardr: cool, thanks
<barry> leonardr: i wonder if things like operation_returns_collection_of() should take strings optionally, or whether some of the apihelpers (e.g. patch_collection_return_type) should be pulled into lazr.restful
<leonardr> barry: a later version of zope has some kind of defered reference which is how we were planning to solve the problem
<barry> cool.  best not to reinvent the wheel!
<statik> yo
<statik> how can i tell who got merge proposal 10000? I got 9999, and herb got 10001
<gmb> Minor query, but why does edge still show "Launchpad 2.2.6"? Do we fail at updating version numbers or something?
<joey> gmb: I asked earlier. mrevell's branch fixes that
<joey> don't know status of the branch though, haven't looked
<gmb> joey: Ah, okay. Ta.
<thumper> statik: you can't right now
<thumper> statik: without a db query :)
<thumper> sinzui: ping
<statik> thumper: ah, ok
<sinzui> hi thumper
<thumper> statik: we were thinking of having a short hand redirect
<thumper> statik: or maybe changing the canonical url
<thumper> sinzui: call?
<sinzui> okay
<wgrant> intellectronica: Oh, the project bugs index isn't scheduled for a redesign? :(
<wgrant> that's one view that really, really needs it.
<intellectronica> wgrant: project, most definitely. project group, yes, but maybe lower priority
<wgrant> intellectronica: Oh, damn, missed that. Sorry.
<intellectronica> or maybe they'll be able to share the design, but project is really so much more important than project groups. there aren't many of them anyway
<wgrant> Yep.
<sinzui> thumper: https://pastebin.canonical.com/21028/
<rockstar> thumper, skype?
<EdwinGrubbs> sinzui: ping
<sinzui> Hi EdwinGrubbs
<EdwinGrubbs> sinzui: can you think of any reason why the pagetests would cause view/+related-pages not to load at all, even though it appears fine when I use "make run"?
<bac> sinzui: queued ping
<sinzui> EdwinGrubbs: That is an interesting scenario. No I cannot think is a clever reason. When I see things like that I assume the test runner is in the wrong tree, or the test machine and devel machine have different branches
<sinzui> hi bac
<sinzui> EdwinGrubbs: is there a URL for your branch?
<bac> sinzui: very quick skype?
<sinzui> bac: I am ready
<motin_0> hey all
<rockstar> thumper, I think that's the shortest standup I've ever participated in that you also participated in.  :)
<motin_0> I have been playing around with launchpad, and got everything to run using the current trunk except for mail features...
<motin_0> I get this:
<motin_0>     2009-08-12T00:06:15 ERROR QueueProcessorThread Error while sending mail from bounces@canonical.com to mymail@gmail.com.
<motin_0>     Traceback (most recent call last):
<motin_0>       File "/home/launchpadtest/launchpad/lp-branches/devel/lib/zope/sendmail/delivery.py", line 203, in run
<motin_0>         unlink(filename)
<thumper> rockstar: :)
<motin_0>     OSError: [Errno 2] No such file or directory: '/var/tmp/launchpad_mailqueue/new/1250028370.11349.machinehostname.1187667112'
<motin_0> is it usual for new launchpad installations?
<motin_0> couldnt find info about it in the wiki
<rockstar> motin_0, it's best to paste tracebacks in a pastebin, so you don't mask what other people are saying.
<motin_0> sorry, didnt realize it was so many lines
<motin_0> http://paste.ubuntu.com/251626/
<rockstar> motin_0, do you have a mailserver running locally?
<motin_0> rockstar: yes, apache can send mail fine. standard ubuntu lamp setup
<EdwinGrubbs> sinzui: lp:~edwin-grubbs/launchpad/team-edit-pages  and the failing test is xx-object-branding.txt
<beuno> sinzui, your project 3.0 branch is amazing
<beuno> I just saw one thing that looks odd
<beuno> the "Does not use Launchpad for development." text
<beuno> seems randomly placed
<sinzui> hmm, that may be bad markup then
<beuno> sinzui, but this is awesome
<beuno> you've outdone yourself
<beuno> I owe you more alcohol than I can afford
<sinzui> beuno: I'll take a look at it. I'm going to add bugs and blueprints to the page after salgado lands his work (because I am reusing his portlets)
<beuno> sinzui, fantastic
<beuno> and the lack of edge updates has actually benefited us
<beuno> the work will be rolled out more complete
<motin_0> during startup I get only one weird message, about mailman (which is mail list managing, not mail sending, right?) http://paste.ubuntu.com/251628/
<beuno> I'm off for a quick run, be back in 30'
<sinzui> I am going to hack on some tales stuff for person links, and the header and footer. I'll start the distro page tomorrow
<thumper> matsubara: is there any way to get the nice oops rendering of https://lp-oops.canonical.com/oops.py for launchpad.dev oopses?
<matsubara> thumper, yes
<thumper> matsubara: how?
<matsubara> thumper, lp:~matsubara/oops-tools/oops-tools-django
<matsubara> thumper, that's the branch you want. There's a README with some setup instructions.
<thumper> matsubara: fantastic, is this on the dev wiki anywhere?
<wgrant> Are lp-dev-utils/lp-qa-tools/oops-tools going to be public at some point?
<matsubara> wgrant, there are discussions about open sourcing lp-dev-utils. there are no plans to make lp-qa-tools/oops-tools public in the short term
<matsubara> thumper, nope. that tool is not used by a lot of people, so I'm keeping docs in the README file
<wgrant> matsubara: OK.
<matsubara> thumper, and since oops-tools is not open source, I guess it's confusing to have docs in the public wiki about it.
<thumper> matsubara: ok
<wgrant> Wow, the new project page is good, if a little crashy.
<wgrant> (everything explodes due to a typo if the project uses blueprints)
<motin_0> hmmm. I'd love to start trying out launchpad for real but the current situation is that emails are not being sent from the server. no error messages in the log, nothing showing up in syslog (as mail usually does when sent from apache/php) - anyone can point me in the right direction to fix this?
<wgrant> motin_0: The emails should be sent to root@localhost, I think.
<wgrant> But if you just want to try out Launchpad (rather than hacking it), why not use staging.launchpad.net?
<sinzui> EdwinGrubbs: The TeamEditMenu (nav menu) is inheriting form TeamOverviewMenu (app menu). I think this is wrong. Move the common links in both to a mixin like I did in product. just searching on branding reveals that we are still defining links many times.  I think we really want to use a mixin to create links, and menus just set which links are used.
<EdwinGrubbs> sinzui: ok
<motin_0> wgrant: I meant hacking it :) I have been using launchpad in ubuntu projects for years
<wgrant> motin_0: Ah. Well, development Launchpads don't send email to the outside world, for obvious reasons. They connect to the local SMTP server and send to root.
<sinzui> EdwinGrubbs: I have seen menu problems several times in the past week. I will update the menu recipe in the conversion document with the kinds of problems we have encountered.
<motin_0> aha wgrant, that explains it :) I wanted to try with a new account, but best practice is, you say, to always use the admin@canonical.com account?
<wgrant> motin_0: Or create a new account, and check the local mailbox.
<wgrant> motin_0: Or use utilities/make-lp-user
<motin_0> wgrant: thank you very much for the info and the suggestions. this was really hard to know without asking someone like you
<wgrant> sinzui: Is the subheading in 3.0 meant to be present if it's identical to the context heading above the new breadcrumbs?
<wgrant> motin_0: I wonder if that bit is documented anywhere on the dev wiki. Let's see...
<motin_0> wgrant: just added it
<wgrant> motin_0: I don't see that change.
<motin_0> wgrant: really slow. still loading the save page
<motin_0> been doing it for several minutes
<motin_0> ah there
<motin_0> https://dev.launchpad.net/FAQ
<wgrant> Ah, good.
#launchpad-dev 2009-08-12
<motin_0> hmm, newbie question I guess, but I have to make changes to the config and try to change it using   "export LPCONFIG=tempconfig; make run" and "LPCONFIG=tempconfig make run" but it keeps starting up with LPCONFIG=development ... a hardcoded restriction somewhere?
<thumper> motin_0: LPCONFIG=tempconfig make run
 * thumper thinks
<motin_0> thumper: that's exactly what I try. it still starts as for instance utilities/shhh.py make -C sourcecode build PYTHON=python2.4 \            PYTHON_VERSION=2.4 LPCONFIG=development
<thumper> motin_0: have you set up a tempconfig?
<motin_0> there is a directory called tempconfig, yes
<motin_0> it that's enough?
<motin_0> a copy of development directory
<thumper> tried `make run LPCONFIG=tempconfig`?
<thumper> I can't remember how to make make work properly
<motin_0> ey thumper that did it :)
<motin_0> so make has it's own way of using env vars
<motin_0> thats new to me, thanks for the suggestion!
<wgrant> Well, they're make variables, not envvars.
<wgrant> They just happen to use an unfortunately similar syntax.
<motin_0> wgrant: ah. in configs/README.txt it says "This directory stores configs. The config used is selected on startup using the LPCONFIG environment variable."
<thumper> motin_0: for scripts yes, not for make
<thumper> kinda confusing
<thumper> patches welcome :)
<motin_0> I thought about it, but then I would probably commit some change that itself was wrong, since I have yet not understood the greater picture
<thumper> spm: can you take a look at the buildbot?  it seems stuck
<motin_0> I can't get sendmailMailer to work. despite it being a development box, I really want it be able to send mail to me and my colleagues that will work on this together... is it hard-coded to prevent this in all cases or can anyone show a config-file that would actually allow external mails?
<spm> thumper: not atm ; am mid a rollout. hopefully in the next 15-30
<thumper> spm: ack
<thumper> motin_0: most development instances are careful not to send real email
<thumper> motin_0: it will be configurable
<thumper> but I'm not sure where it it
<motin_0> thumper: right, I've been following zope docs and that won't cut it, so I guess launchpad devs has taken some extra measures
<thumper> motin_0: almost certainly
<thumper> ./configs/development/launchpad-lazr.conf:211:default_recipient_address: root@localhost
<motin_0> yeah I can configure the stub-mailer
<thumper> ./configs/development/launchpad-lazr.conf:210:default_sender_address: root@localhost
<motin_0> to send to a specific address
<motin_0> hmm, I could hack stub-mailer...
<thumper> you shouldn't need to hack IMO
<motin_0> I never got farther than this using zope example confs: ConfigurationError: ('Unknown directive', u'http://namespaces.zope.org/mail', u'sendmailMailer')
<motin_0> feel like hacking is the only way to go at this stage
<motin_0> hmm I think I found something
<motin_0> I'll get back with results
<thumper> look in mail-configure-normal
<motin_0> thumper: yep, problem is that there is no mailer named sendmail
<motin_0> and defining it by means of standard zope yields the above error
<motin_0> hmm seems I have been looking in to old docs or something
<motin_0> http://mail.zope.org/pipermail/zope3-checkins/2005-July/025405.html
<motin_0> sendmailMailer was removed in 2005
<motin_0> sorry about that
<maxb> Has anyone else observed lots of garbage-warnings in the lp tests, concerning bzrlib.smart.medium.SmartSimplePipesClientMedium ?
<motin_0> thumper: yep, that solved it. sendmailMailer is dead end.   smtpmailer is the way to go. simple as that. just some bits of doc-info (in the code) left still referring to sendmailMailer made me go down the wrong path here
<motin_0> thanks for your support thumper
<thumper> motin_0: we have been using quite an old zope
<thumper> motin_0: we are actively updating it now
<motin_0> ok that might explain some discrepancy
<spm> thumper: have kicked the slaves; if that still fails to shake tailfeathers, I'll try the master as well
<thumper> spm: thanks
<thumper> spm: you might need to give the master a shove
<thumper> spm: the db_lp builder and the bzr builder are stuck
<spm> yeah was coming to that conclusion too...
<thumper> spm: and the lp builder hasn't started again :(
<spm> see if that does it
<spm> yay, that looks like it
<thumper> sinzui: I think your branch may have problems
<thumper> sinzui: my ec2test has some weird failures that seem unrelated to my 3.0 changes
<sinzui> thumper: what are the errors
<thumper> sinzui: https://pastebin.canonical.com/21035/
<sinzui> thumper: that is me,
<sinzui> thumper: Those are the links I had to add to the menu, so I will update the test.
<thumper> sinzui: ok
<sinzui> thumper I recall the test and thought I fixed it.
<thumper> sinzui: did you commit?
<sinzui> thumper: I was very tempted to remove the SimpleViewClass part of the test, maybe even the name of the template. Do we care about the link or the specifics of the implementation?
<sinzui> thumper: I think I did something very stupid, revert, because I really think this test is stupid
<thumper> sinzui: delete the test then
<sinzui> thumper: The check_menu_links link appears to be do some very cumbersome work to prove the links works.
<thumper> sinzui: check_menu_links isn't really adding anything here
<thumper> sinzui: it should really just validate the links
<thumper> sinzui: we don't care if people add some
<thumper> sinzui: we do care if they are wrong though
<thumper> perhaps a new unit test that validates the menu
<sinzui> I think that was the intent. EdwinGrubbs struggled with this test a long time ago. yes, a unitest that verifies every menu is the correct answer.
<thumper> sinzui: I could add in the extra bits into the branch I'm about to land if you like
<thumper> sinzui: or remove that chunk
 * thumper wonders about the testfix demon
<sinzui> thumper: adding chunks is my immediate intent.
<thumper> ok
<thumper> sinzui: I can do that if you like
<thumper> sinzui: or should we wait for the failure and testfix?
<sinzui> thumper: There are some equally idiotic translations menu tests I want to read before doing something clever like, get every menu, instantiate the link for the menu's context, and report any named adaption failures
<sinzui> I just branched. I can put the test fix in in a few minutes.
<sinzui> Oh! PQM is still frozen
<thumper> sinzui: if you put testfix in, will it preempt testfix mode?
<thumper> spm: unfreeze pqm?
<spm> thumper: oops. will do.
<thumper> sinzui: ok, I'll leave that bit to you then
<thumper> sinzui: I'm about to land generic-edit.pt
<sinzui> sweet
<thumper> sinzui: already deleted half a dozen redundant page templates
<sinzui> thumper: product-menus is no longer brittle. check_menus reports adaption errors, not state. It is now a good candidate for a real test that verifies every menu.
<thumper> sinzui: cool
<thumper> need a review?
<sinzui> No, not for this
<thumper> ok
<sinzui> but I think I need to talk to salgado about how to get every menu in module without instantiating it.
 * thumper helps with dinner, back later
<jtv> hi folks
<spm> hey jtv
<jtv> hi spm
<adeuring> good morning
<gmb> Morning Launchpadders.
 * thumper is back wrestling with +activereviews
<mrevell> Hi Launchpad-land
<bigjools> good morning Wolves
<thumper> hi uk poeoples
<thumper> ETOOLATE
<bigjools> eyup thumper
<gmb> Err.
<gmb> Anyone have any idea why I'm getting "ImportError: No module named buildout.buildout" this morning when I run make
<gmb> ?
 * gmb re-runs rf-get for giggles
<thumper> gmb: I think gary landed an updated buildout egg
<gmb> thumper: Yeah, looks that way. Running bin/buildout manually seems to have fixed it.
 * gmb hates at buildout at sub-11-am
<thumper> ah crap
<thumper> I need to change the MenuAPI tal formatter for my code to work nicely :(
<bigjools> thumper: would it be easy to use the webservice api to check for new revisions, but only for an lp sub-project? (soyuz in my case of course)
<thumper> no
 * bigjools 's idea goes down in flames
<thumper> bigjools: it would be quite hacky bzrlib
<bigjools> thumper: I will talk to diogo, they do it for the qa pages
<bigjools> unless it's a dirty hack based on who landed it...
<wgrant> It is.
<wgrant> I saw the code once.
<wgrant> And contributors appear on LaunchpadTestPlan.
 * maxb wonders: what in Launchpad uses lxml / pysqlite2 
<maxb> Because I've not rebuilt those to be available with python 2.4 in karmic, yet nothing's breaking
<wgrant> maxb: sqlite is probably bzr-git.
<gmb> ISTR we wanted lxml to die a horrible flaming death a while back, but I might just be projecting.
<wgrant> Isn't lxml the XML lib that *shouldn't* die?
<wgrant> lazr.restful's tests use lxml, but grep doesn't show anything else.
<maxb> maybe it's just gratuitously included for people who hack on lazr.restful then
<wgrant> Probably.
<maxb> Oh well. When I get through another full test run tonight, we get to see how many of the problems I'm seeing are due to running on Karmic, vs. how many due to Python 2.4 - because even running Py2.4 on karmic, the testsuite is not entirely happy
<maxb> If we're lucky it'll just be the "it breaks if the system bzr is new enough for buildout to use it" problem
<bigjools> we usually get lots of breakage when moving to a new distroseries
<wgrant> Particularly in the Soyuz bits that use dpkg bits?
<bigjools> that's rare actually
<deryck> Morning, all.
<bigjools> hi deryck
 * gmb waits has firefox tries to do > 1 thing at once and melts into a steaming pile of XUL.
<wgrant> gmb: This is why I'm starting to like Chromium.
<wgrant> Although LP bug listings are a bit broken in it.
<gmb> wgrant: Yeah, I'm very much of the same school. Haven't seen the bug listing problem yet... can you point me to an example?
<wgrant> gmb: The importance icons are absent.
<gmb> Oh. That's suboptimal.
<bigjools> wgrant: does chromium have anything like firebug, noscript and adblock?
<intellectronica> wgrant: you mean the sorting? i thought it was fixed
<intellectronica> bigjools: it has tools equivalent to firebug built in. i think extensions are now possible but there isn't that much choice out there yet
<bigjools> okay - ta
<bigjools> seeing pages now without adblock is....painful
 * intellectronica actually reads the scrollback
<wgrant> There's an adblock-like thing around now, apparently.
<bigjools> I will stick with Konqueror as my alternative browser
<wgrant> But it's all still a bit unfinished.
<wgrant> And does strange focus things.
<intellectronica> wgrant: so, there are still quite a few problems with sprites in webkit-based browsers. if you find anything it would be great if you can file a bug (or at least give a shout here) because little icons are so easy to miss (and you can't test that automatically either)
<wgrant> intellectronica: I've noticed a few. I'll file bugs in future.
<intellectronica> thanks
 * gmb lunches
<beuno> cprov1, bigjools, call in 15?
<bigjools> yep
<cprov> ok
<gmb> 14
<gmb> Rats
<sidnei> anyone from bugs around?
<sidnei> gmb: ?
<gmb> sidnei: Howdy. What can I do for you?
<sidnei> gmb: so, i think we found a bug with 'super projects' or whatever is the term for that
<sidnei> gmb: we have landscape-project, which is a super-project for landscape and landscape-client
<gmb> Right.
<sidnei> gmb: if someone is an admin on '~landscape' and files a bug directly into 'landscape', he gets all the extra options
<sidnei> gmb: if this person files a bug in 'landscape-project', but picking 'landscape' from the drop-down of 'child' projects, he only gets 'tags' in extra options
<bigjools> beuno: skype?
<gmb> sidnei: Right. I *think* this is something we're kind of aware of. Let me just take a look...
<sidnei> gmb: ok
<beuno> bigjools, yes, searching for headphones
<gmb> sidnei: So, this is essentially bug 375144. The problem is that we work out who can see the extra options based on what context they're filing the bug in (i.e. project, project group, front page, etc.) rather looking at what they're trying to file the bug against. I'll update the bug to include this case.
<mup> Bug #375144: "Extra options" section doesn't show for my projects on the frontpage +filebug form, even though I'm the bug supervisor <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/375144>
<gmb> sidnei: Thanks for the heads-up.
<sidnei> gmb: cool, thank you!
<bigjools> beuno: I'm surprised you need to search, given the number of calls you're on ;)
<beuno> bigjools, it's usually my way of saying "I'm going to get some tea"
<beuno> bigjools, what's your skype id?
<bigjools> beuno: I'll give you no guesses
<beuno> bigjools, cprov, ready when you are
<bigjools> beuno: noodles is joining us too, to listen in
<bigjools> I guess I'l host
<cprov> beuno: https://dev.launchpad.net/SoyuzBuildIndexPage
<beuno> cprov, https://devpad.canonical.com/~beuno/LP_new_design_Bugs_v3.1.png
<henninge> bigjools: can a user not remove his/her own ppas?
<bigjools> henninge: they can't, they need to file a question
<henninge> bigjools: ok, assign it to you, then?
<bigjools> and the losas will take care of it
<henninge> ah, losas
<bigjools> no, losas
<henninge> ;)
<henninge> cheers
<bigjools> np
<beuno> cprov, https://devpad.canonical.com/~beuno/LP_new_design_Home.png
<noodles775> http://people.canonical.com/~michaeln/3-0-builder-mechanical/0-builder-index-v2.png
<bigjools> someone please shovel more coal into staging
<cprov> http://people.canonical.com/~ed/dsp_mockup2.png
<bigjools> https://dev.launchpad.net/VersionThreeDotO/Soyuz/NavigationRedesignUI
<bigjools> https://edge.launchpad.net/ubuntu/+source/firefox-3.5
<bigjools> https://dev.launchpad.net/SoyuzPackageUI?action=AttachFile&do=get&target=sample_layout_for_ff.jpg
<henninge> beuno: ping
<beuno> henninge, hi
<henninge> hi beuno!
<henninge> beuno: We'd like to have a call/chat with you about the translate page that I made suggestions for.
<henninge> beuno: we have a new mock-up! ;_)
<beuno> henninge, wooo
<henninge> beuno: When would be a good time?
<beuno> I'm on the phone with the soyuz guys
<beuno> and have 3 more calls today
<beuno> how's your schedule?
<henninge> beuno: tomorrow suits you better then, I guess.
<beuno> henninge, maybe today as well
<henninge> beuno: meetings, meetings, meetings ... let's see if I can fit you in ...
<henninge> ;-)
<beuno> henninge, whats your schedule today?
<henninge> beuno: reviewer's meeting in 7 minutes. And I have to leave at aout 17 UTC at the latest.
<henninge> danilos: how's your schedule today?
<danilos> henninge: I am up for a call anytime except when beuno is on a call as well later in the evening :)
<beuno> henninge, danilos, I'll try to in 30 minutes
<henninge> beuno: cool, fine by me.
<barry> hello fellow launchpad hackers!  we'll be having our weekly ameu reviewers meeting in #launchpad-meeting in about 4 minutes
<barry> reviewers -> #launchpad-meeting in 1m
<beuno> bigjools, https://devpad.canonical.com/~beuno/LP_userdetail.png
<bigjools> barry: soyuz people will be late, sorry
<jmux> Hi - I downloaded and installed LP on my Debian Lenny Server. I'm already using sbuild and schroot. Now I'm looking to integrate everything with builldd, but couldn't find any information. Anyone here, who can point me to some docs?
<jmux> I saw there is an old sbuild package and some scripts in the lib/canonical/buildd directory, but it doesn't include a real buildd and the control file of the generated launchpad-buildd package doesn't depend on buildd, so I'm a little bit lost here.
<barry> bigjools: no worries.  looks like it might be a short meeting anyway
<bigjools> http://people.canonical.com/~ed/dspr_mockup.png
<bigjools> https://dev.launchpad.net/SoyuzPackageUI?action=AttachFile&do=get&target=package_release_details.jpg
<sinzui> gary_poster: I have two machines reporting ImportError: No module named buildout.buildout
<gary_poster> sinzui: doing what?
<barry> sinzui, gary_poster oh noes!  i just hit the same error doing an rf-get
<sinzui> gary_poster: after rocketfuel-get and when I run make build in a pristine branch
<gary_poster> sinzui: so (1) rocketfuel-get, (2) rocketfuel-branch?  or (1) rocketfuel-get, (2) bzr branch trunk foo (3) cd foo (4) link-external-dependencies (5) make?
<gary_poster> (the first route works for me)
<cprov> jmux: documentation isn't good for lp-buildd, can you please send an email to lp-users ML asking for more info ?
<barry> gary_poster: for me, #1
<gary_poster> barry: bin/buildout is the workaround apparently, but I need to know why you guys are seeing this
<barry> i.e. just rocketfuel-get
<gary_poster> uh.
<barry> gary_poster: i have a few minutes to debug this with you before i have to head out for a little bit
<sinzui> 2) is bzr branch rocketfuel pristine; cd pristine; make build
<gary_poster> sinzui: you are not linking (my step 4)?
<sinzui> gary_poster: My script has always done that
<gary_poster> barry: ok (I'm also trying to help mthaddon in another channel)
<barry> gary_poster: k
<jmux> cprov: Ok - will do
<gary_poster> sinzui: ok.  I was just trying to understand what you were clarifying for me in your message.  Were you simply identifying that you were doing the "raw" approach, not rocketfuel-branch?
<cprov> jmux: cool, thanks.
<sinzui> gary_poster: yes, I have always linked
<gary_poster> cool
<gary_poster> ok barry
<gary_poster> so in bin/buildout
<sinzui> gary_poster: I do see zc.buildout-1.4.0dev_gary_r102684-py2.4.egg/zc/buildout/buildout.py
<gary_poster> what is the buildout egg linking to?
<gary_poster> (in your sys.path)
<gary_poster> sinzui: does make work now?  i.e., is this a "do it twice and it works" sort of thing?
<barry> gary_poster: '/home/barry/projects/launchpad/eggs/zc.buildout-1.4.0dev_gary_r102684-py2.4.egg',
<gary_poster> barry: and does that exist?
<sinzui> NO it does not, I did that before asking for help
<beuno> henninge, danilos, give me a little bit to recover from an hour of soyuz, and I'll be right with you
<barry> gary_poster: yep
<bigjools> the cheek
<henninge> beuno: we feel for you ...
<gary_poster> barry: could you pastebin me the error, with just maybe 10 lines above it?
<noodles775> c'mon henninge, I've been looking through some translations code ;)
<danilos> beuno: sure, I need to a minute as well
 * bigjools is laughing a lot
<barry> https://pastebin.canonical.com/21051/
<barry> gary_poster: ^^
<Knut-HB> Hi, it's me again with a problem with make schema (I've been tasked to install a local launchpad on a Jaunty server version): http://paste.ubuntu.com/251949/ everything up to "make schema" is just fine and quits with no errors but make schema quits with this error. After installing Ubuntu server I installed the available updates and the latest kernel.
<sinzui> barry: gary_poster: That is the error I am getting
<henninge> noodles775: and, weren't you amazed at the beauty and simplicity ???
<noodles775> ;)
<sinzui> Knut-HB: I do not think you are the owner of your database
<danilos> beuno, henninge: ok, I am ready whenever you are
<maxb> gary_poster: I think I have test failures that may be due to using the system bzr - does your just-landed branch mean lp will always use a private bzr egg, or do I have to poke some buildout config somewhere to make it do that?
<gary_poster> barry: does the egg look broken somehow?  so, you can try ls /home/barry/projects/launchpad/eggs/zc.buildout-1.4.0dev_gary_r102684-py2.4.egg/zc/buildout and see if there is a buildout.py and an __init__.py
<henninge> danilos, beuno: I am ready, too.
<barry> gary_poster: it looks fine!
<gary_poster> maxb: the branch will make it use the private egg.  You will need to merge and rerun bin/buildout (or do a make clean and make)
<maxb> where the branch == now on devel, right?
<Knut-HB> sinzui: http://paste.ubuntu.com/251950/ i just got through the steps mentioned at https://dev.launchpad.net/Getting and https://dev.launchpad.net/Running
<danilos> henninge: where's the latest mockup (or mockups) we want to discuss?
<gary_poster> barry: try cd /home/barry/projects/launchpad/eggs/zc.buildout-1.4.0dev_gary_r102684-py2.4.egg and then start up python2.4 and try importing zc.buildout.buildout?
<beuno> danilos, henninge, I'll be ready in 15ish
<gary_poster> maxb: yes
<danilos> beuno: ok, don't forget to wipe your a... uhm, whatever :)
<maxb> excellent, I'll fire up another full testrun overnight and see what happens :-)
<henninge> danilos: still https://devpad.canonical.com/~henninge/mockups/translate-page/list.html
<henninge> danilos: just one
<sinzui> Knut-HB: try this to ensure you have the necessary permissions: `sudo -u postgres createuser -s -d $(id -un)`
<barry> gary_poster: happy as a clam:
<barry> @mission[[~/projects/launchpad/eggs/zc.buildout-1.4.0dev_gary_r102684-py2.4.egg:1012]]% /usr/bin/python
<barry> Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41)
<barry> [GCC 4.3.3] on linux2
<barry> Type "help", "copyright", "credits" or "license" for more information.
<barry> >>> import zc.buildout
<barry> >>> import zc.buildout.buildout
<barry> /usr/lib/python2.6/dist-packages/Pyrex/Compiler/Errors.py:17: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
<barry>   self.message = message
<barry> >>>
<barry>  
<Knut-HB> sinzui: createuser: creation of new role failed: ERROR:  role "launchpad" already exists
<gary_poster> maxb cool :-)  also, my zbuildout branch has all tests passing (though it uses some eggs I have not shared yet).  I'm pushing some more of our changes upstream and then will be getting it all reviewed
<sinzui> Knut-HB: did you have pg installed before you started to setup launchpad?
<Knut-HB> sinzui: don't think so. as mentioned above: i just did what the two pages told me to do
<gary_poster> barry: urgh.  so why don't you remove shhh.py from before the problem line in the Makefile and see if it happens to give us more info when you run make again?
<maxb> my current objective is to actually accomplish a clean test run using python 2.4 so I have a baseline to test branches on so I can submit them with a clear conscience :-)
<barry> gary_poster: sure thing.  actually 'make SHHH= build' shoudl do the same thing
<sinzui> Knut-HB: okay, I think you would know if postgesql was installed
<gary_poster> maxb: :-) +1
<gary_poster> barry: sure, cool
<barry> after a make clean.  this crashes early
<barry> % make SHHH= build
<barry> scripts/update-bzr-version-info.sh
<barry> Creating bzr-version-info.py at revno 9098
<barry> python2.4 bootstrap.py --ez_setup-source=ez_setup.py \
<barry> 		--download-base=download-cache/dist --eggs=eggs
<barry> Creating directory '/home/barry/projects/launchpad/devel/bin'.
<barry> Creating directory '/home/barry/projects/launchpad/devel/parts'.
<barry> Generated script '/home/barry/projects/launchpad/devel/bin/buildout'.
<barry> ./bin/buildout configuration:instance_name=development -c buildout.cfg
<barry> Traceback (most recent call last):
<barry>   File "./bin/buildout", line 17, in ?
<barry>     import zc.buildout.buildout
<barry> ImportError: No module named buildout.buildout
<barry> make: *** [/home/barry/projects/launchpad/devel/bin/py] Error 1
<Knut-HB> sinzui: you mean "postgresq"?
<gary_poster> barry: will try a make clean locally 1 sec
<henninge> beuno: I just updated that mockup, it unfolds plurals now, too.
<henninge> danilos: ^
<henninge> sorry beuno
<gary_poster> barry: works fine after make clean.  will try a fresh eggs dir.
<danilos> henninge: cool, let me take a look
<sinzui> Knut-HB: PostgreSQL is the relational database that Launchpad uses. rocketfuel-setup installs it if you do not already have it. Most db issues encountered after setting up launchpad relate to a previous installation of PostgreSQL, which you do not have.
<danilos> henninge: one immediate suggestion -- icon should be on the top right, not after the entire text
<danilos> henninge: a simple floater should do, I believe :)
<gary_poster> barry: works fine. :-( ok...
<henninge> danilos: hm, I used a table, not knowing what else to do, to get the radio button where it is.
<Knut-HB> sinzui: rocketfuel-setup did NOT install postgresql, i post-installed just a moment ago after you mentioned it
<gary_poster> barry: echo PYTHONPATH?
 * henninge tries the floater
<sinzui> Knut-HB: You make need to follow the instructions here: https://dev.launchpad.net/DatabaseSetup
<barry> gary_poster: % echo $PYTHONPATH
<barry> /home/barry/env/python
<gary_poster> $PYTHONPATH I should say :-)
<mrevell> kfogel: ping
<barry> gary_poster: been that way for a bazillion years
<Knut-HB> sinzui: ok thx, i check that page out
<kfogel> mrevell: pong
<danilos> henninge: right, implementation details is something we can figure out later, I am talking about green/red/whatever icon for longer texts
<henninge> danilos: can I float a single <img> or do I need to put in a <div>?
<danilos> henninge: you can float an image
<danilos> henninge: fwiw, img was the first one to allow floating in html :)
<gary_poster> barry, let's hack bin/buildout.  Please have it pretty print the pythonpath variable and sys.path, and let's see what they look like on a pastebin.
<danilos> henninge: using <img float="...">, but you should use CSS, of course :)
<henninge> danilos: oh right, I remember
<henninge> danilos: but it was <img align="..">, wasn't it?
<danilos> henninge: I think it was actually float, but I am not sure :)
<danilos> henninge: guess it was align actually
<gary_poster> barry: also, would be good to know if the bin/buildout workaround that allenap and gmb reported works for you.  maybe start another branch and give it a try?  or sinzui, have you already tried that?
<sinzui> gary_poster: I see sane building now
<barry> gary_poster: https://pastebin.canonical.com/21053/
<gary_poster> sinzui: you mean after the bin/buildout thing?
<henninge> danilos: yeah, and the old vspace and hspace attributes ...
<sinzui> yes
<barry> what's the bin/buildout thing?
<danilos> henninge: nothing beats bgcolor, but anyway... :)
<sinzui> barry: run  bin/buildout; make build;
<sinzui> barry: Run that after linking
<gary_poster> barry: allenap and gmb reported on the list that if you run bin/buildout this symptom magically goes away.  yeah, what he said. :-)
<barry> gary_poster, sinzui bin/buildout does seem to be running just fine
<gary_poster> barry: ok, maybe we are getting somewhere now.  let's try clearing out PYTHONPATH in the Makefile so we don't pass anything to it
<barry> oops.  i just ran bin/buildout :(
<barry> gary_poster: and make build is happy.
<barry> gary_poster: and i have to go afk for a little while :(
<gary_poster> barry: well, at least you are unblocked.  Try another rocketfuel-branch...ok
<barry> gary_poster: happy to continue debugging this with you when i return
<gary_poster> sinzui: may I use you for debugging now?
<gary_poster> if not I will take barry up on his kind offer
<gary_poster> sinzui: I wanted to see if rocketfuel-get and rocketfuel-branch worked for you now
<sinzui> gary_poster: I have never used the later
<gary_poster> sinzui: no need
<gary_poster> sinzui: just want to know if what you did before magically works now
<gary_poster> so you can recast my request as appropriate
<sinzui> I can run rocketfuel-get. I ran it on both my computers in the last 45 minutes
<gary_poster> could you retry the thing that broke, then?
 * sinzui runs rocketfuel-get
<Knut-HB> sinzui: http://paste.ubuntu.com/251967/ i did everything https://dev.launchpad.net/DatabaseSetup told me to do (both ways) and it still does not work
<gary_poster> sinzui: oh...so, you mean, rocketfuel-get broke a while ago, but has since been working?
<sinzui> gary_poster: I mean I ran that, I got the buildout error because the script tries to make all the branches in the repo
<gary_poster> OIC
<sinzui> Knut-HB: I do not know what else to try. stub will be online in 9 hours. He has the most experience with the database.
<danilos> beuno: ok, it can't be that bad :)
<sinzui> gary_poster: rocketfuel-get completed successfully.
<beuno> danilos, henninge, I'm ready
<gary_poster> sinzui: ...ok.  well, on the bright side, I guess that fixed it.  On the not-so-bright side, I don't know why.  I'll send out a note to the list to that effect.  thank you.
<danilos> beuno, henninge: https://devpad.canonical.com/~henninge/mockups/translate-page/list.html
<Knut-HB> sinzui: ./utilities/launchpad-database-setup root and sudo make schema and then make run: everything is ok, everything is fine and launchpad runs locally
<sinzui> Knut-HB: I will did not know about that script. I will remember it
<Knut-HB> sinzui: now the question is: why does everything work as sudo but not as user "launchpad"?
<beuno> https://translations.edge.launchpad.net/ubuntu/jaunty/+source/gnome-games/+pots/gnome-games/es/+translate
<beuno> danilos, henninge, ^
<sinzui> Knut-HB: I do not think you own your database
<bigjools> Knut-HB: sinzui: there's a problem with accounts called "launchpad", it clashes with a permission in security.cfg
<bigjools> I found this out the hard way
<Knut-HB> bigjools: you're not serious? Oo we tried to solve this problem in the last 24h Oo
<bigjools> very serious, unfortunately
<bigjools> I had to rename my local account, and fix the db permissions so it can be a PG superuser
<bigjools> there's a role in security.cfg called "launchpad" which will overwrite your superuser when you run "make schema"
<Knut-HB> bigjools: then there should be a warning at dev.launchpad.net that the user has not to be named "launchpad"
<bigjools> Knut-HB: yes, that would be a good idea
<Knut-HB> bigjools: or at least an entry in the FAQ
<danilos> henninge, beuno: one big advantage of a two column layout: headings like 'English' and 'Translation' are not repeated with every message
<henninge> danilos: yeah, that was one of my issues, too.
<danilos> henninge, beuno: so, I'd still like us to experiment with both, I'll try to figure out how we can best do user testing
<beuno> danilos, sure. I think it won't work, but knock yourself out!
<danilos> beuno: I was quite skeptical at the beginning, but the more I think about it, the more I am in favour of it
<beuno> danilos, that's the problem with UI. Thinking only gives you reasons to do it  ;)
<danilos> beuno: heh, right :)
<beuno> danilos, I think that if the first impression is disconcerning, it usually means it won't work
<danilos> beuno: yeah, it usually means that the learning curve is great :(
<danilos> henninge: do keep the existing mockup as the two column layout one, we do want to be able to compare the two :)
<henninge> danilos: ok
<henninge> danilo-bbl: hey, wait!
<danilo-bbl> henninge: what? I am hungry... :)
<henninge> chasing me through channels ...
<henninge> danilo-bbl: how long will you be gone?
<henninge> danilo-bbl: I'll probably leave in like 45 minutes
<danilo-bbl> henninge: half an hour or so... I guess we can talk tomorrow then
<henninge> danilo-bbl: ok, let's do that. Enjoy the food.
<danilo-bbl> thanks
<henninge> gary_poster: hi!
<gary_poster> hi henninge :-)
<henninge> gary_poster: It seems I have not yet used any other script than ec2test from lp-dev-utils.
<henninge> gary_poster: at least disable-projects complains about launchpadlib missing.
<gary_poster> me either, though apparently it has some scripts we are supposed to be using certain ones for CHR
<gary_poster> henninge: try using bin/py from a launchpad trunk
<henninge> gary_poster: oh, I thought you were doing those, too.
<henninge> gary_poster: ok, I'll try that.
<gary_poster> no :-)
<henninge> ;-)
<henninge> gary_poster: yup, works
<gary_poster> cool
<henninge> thank you
<mrevell> speak to you tomorrow guy
<mrevell> s
 * gmb EoDs
<barry> gary_poster: it's back :(
<gary_poster> barry: good for me, bad for you. :-)  OK, so back to what I was going to try last time
<gary_poster> one sec, let me get a patch
<barry> ;) k
<gary_poster> barry: ok, so in this diff, try applying just the Makefile bit (you can do the rest too if you like but it is unrelated) : https://pastebin.canonical.com/21073/
<gary_poster> barry: my hypothesis is that the new ability I added to pay attention to PYTHONPATH suxors as an idea
 * barry applies
<gary_poster> barry: so I haven't actually tested if this masks the Makefile PYTHONPATH and your PYTHONPATH as I hope.  so we'll have to verify (1) does this do what I hope and (2) does it help.  not necessarily in that order.
<barry> #2 is definitely true!
<gary_poster> woo hoo!
<barry> make it so, cap'n
<gary_poster> barry: ok, then that's good enough.  want to give me a review of that diff?  I can switch to the review channel and then explain the other two bits (bug 389065) and then I'll push it
<mup> Bug #389065: system installed eggs overriding buildout provided packages <Launchpad Foundations:In Progress by gary> <https://launchpad.net/bugs/389065>
<gary_poster> or maybe no explanation is needed with that bug :-)
<gary_poster> barry: oops 1 sec
<barry> gary_poster: ping with an unrelated question
<gary_poster> barry: pong with a cucumber
<barry> gary_poster: what about a pointed stick?
<gary_poster> ow!
<barry> :)
<barry> gary_poster: so, what do you think the best way to determine whether the appserver has definitely started up?
<barry> gary_poster: best approach so far, is try to hit the root url for a little while and timeout if it doesn't happen
<barry> gary_poster: but that makes for a fragile-ish test, especially because what i really want to test for is that the appserver has /not/ started up
<thumper> sinzui: I found an area where I'm duplicating queries used in menus
<gary_poster> barry: hm, interesting.  you could try following the log, assuming there is one to follow.  oh...ok.  testing negatives is hard.  um.
<barry> gary_poster: yeah
<thumper> sinzui: I want to be able to get non-link attributes from menus
<barry> gary_poster: check a pid file perhaps?
<gary_poster> barry: could you verify that some error message shows up?
<barry> gary_poster: in a log file you mean?
<gary_poster> barry: right
<gary_poster> if it were possible that would convert it into a positive test, which would be nicer
<barry> yep
<gary_poster> barry: test-appserver-layer.log?
<barry> hmm, okay, let me think about it.  i was just looking for any thoughts that came immediately to mind
<gary_poster> barry: there's a canonical.lazr.pidfile hanging around too
<gary_poster> the log sounds most interesting to me, until/unless it proves unworkable
<barry> gary_poster: thanks!  i might go with the pid file though since it's easy to check the file's existence and contents
<salgado> beuno, what would you expect as the breadcrumbs for breadcrumbs for bugs.lp.net/ubuntu/+source/f-spot/+bug/34074?
<mup> Bug #34074: F-Spot does not warn user if the free space is low before importing photos <F-Spot:New> <f-spot (Ubuntu):Triaged> <https://launchpad.net/bugs/34074>
<salgado> beuno, "Ubuntu > F-spot > F-spot bugs > Bug 34074"?
<mup> Bug #34074: F-Spot does not warn user if the free space is low before importing photos <F-Spot:New> <f-spot (Ubuntu):Triaged> <https://launchpad.net/bugs/34074>
<beuno> salgado, YES. Perfect.
<Ursinha> flacoste, bug 405327
<Ursinha> grr
<Ursinha> https://bugs.edge.launchpad.net/malone/+bug/405327
<barry> leonardr: ping
<leonardr> barry, hi
<barry> hi leonardr.  i'm a little confused as to what's going on with lazr.restful version skew.  it seems like the version we have in sourcecode is fairly different than the one in lp:lazr.restful.  do you know what's up with that?
<leonardr> barry: yes
<leonardr> we cannot make launchpad use a recent version of lazr.restful until gary_poster makes some buildout-related changes
<barry> leonardr: gotcha.  but the intent is that we're run lp:lazr.restful at some point (soonish)?
<leonardr> that is the intent, but i don't know when it'll happen. gary keeps having to do other stuff
<gary_poster> yes
<gary_poster> the branch tests passed two days ago!  yay!  so light is appearing at end of tunnel
<leonardr> hoorah
<barry> okay cool.  i have a couple of convenience changes that would help with my branch's testing.  i will work on getting them into lp:lazr.restful in the hopes that it will show up in launchpad soon
<barry> gary_poster: yay!
<barry> thanks guys
<beuno> sinzui, re: bug 372925      "wooooooo"
<mup> Bug #372925: Change all pillar/person links to go to the overview page on http://launchpad not subdomain.launchpad <navigation> <Launchpad Foundations:In Progress by sinzui> <https://launchpad.net/bugs/372925>
<sinzui> beuno: I am hacking on the registered info in the header now
<sinzui> I think I should try to do the footer too
<thumper> sinzui: can I have a pre-impl call about menus?
<sinzui> sure
 * sinzui gets head phones
<sinzui> thumper: I am reasy
<thumper> sinzui: skype is trying to tell me you aren't on line
<thumper> sinzui: can you call me?
<barry> leonardr: ping again ;)
<leonardr> bary, hi
<barry> leonardr: hi.  zope.schema.Choice() can take a string or enum as its vocabulary argument.  if it's a string (naming an email) lp works just fine, but lazr.restful cannot find the component to marshal.  known bug?
<barry> leonardr: i found bug 364596 it's related but different
<mup> Bug #364596: Choice classes, and their values, should be published as resources <lazr.restful:New> <https://launchpad.net/bugs/364596>
<salgado> sinzui, I completely missed the fact that the new content I added to project-details was inside the <dl>'s div. I moved it out and it looks much better now
<sinzui> salgado: rock
<sinzui> land it
<leonardr> barry, i think i've encountered that bug but didn't know it worked for launchpad. can you give more details about what string you pass in?
<barry> leonardr: hang on, submitting a bug on this...
<salgado> sinzui, not yet; first I'll have a hard time fixing tests
<barry> leonardr: bug 412726 see if that makes sense
<mup> Bug #412726: Choice()'s can take a string for their vocabulary, but lazr.restful can't handle this <lazr.restful:New> <https://launchpad.net/bugs/412726>
<leonardr> barry, can you put up part of the stack trace that shows where the problem is within lazr.restful?
<barry> leonardr: sure, let me add that as a comment to the bug
<barry> leonardr: added
<sinzui> beuno: I have a concern about showing the registered information in the header. The design shows it next to the pillar or person. bugs and branches are under pillars, so where do we show their registered information?
<beuno> sinzui, top right, on the row of the tabs, like in: https://devpad.canonical.com/~beuno/LP_new_design_Bugs_v3.1.png
<sinzui> beuno: okay, that design is far enough from the pillar...but there is still a line between it and the title of the bug.
<sinzui> beuno: should I JFDI and we will see how it works on edge?
<beuno> sinzui, I don't understand what the problem is, but you've solved everything fantastically well up to now, so go for it
 * sinzui does
<sinzui> beuno: I guess I am doing the footer too
<thumper> flacoste: are we doing our call or skipping it?
<thumper> rockstar: sync call?
<rockstar> thumper, yup.
<EdwinGrubbs> spm: is there anything going on in production that would cause timeouts on https://code.launchpad.net/~launchpad/
<rockstar> EdwinGrubbs, known issue, we're talking about it now.
<EdwinGrubbs> rockstar: is it a known production issue or a problem with the page?
<rockstar> EdwinGrubbs, known production/page issue.  Basically, we're adding all the source package branches.
<thumper> in the last 37 minutes we have 200 new branches
<thumper> of which 2 are project branches
<thumper> rest are package branches
<thumper> this is causing growing pains in some branch queries
<beuno> sinzui, :)
<thumper> EdwinGrubbs: the production problem is too many new branches and a badly performing query
<barry> um, wtf? bzr: ERROR: Public branch "bzr+ssh://bazaar.launchpad.net/~barry/launchpad/325962-makestart" lacks revision "barry@canonical.com-20090812214506-vocuvms42g9en1ow".
<barry>  
<barry> yay for bzr push --overwrite
<rockstar> Why do we have lp.answersbugs ?
<thumper> rockstar: it should be lp.coop.answerbugs
<rockstar> thumper, excuse me.  I meant "Why do we have lp.coop.answerbugs ?"
<rockstar> thumper, what I mean to say is, bugbranches and specbranches aren't in lp.coop - putting it where it is now made it hard to find.
<thumper> rockstar: bugbranch and specbranches should be in lp.coop
<rockstar> thumper, okay.  So maybe we need a bug for that.
<rockstar> ...or is there a bug already...
<wgrant> Maybe the bug needs to be moved to the launchpad-coop project?
<rockstar> wgrant, well, it at least needs affect the project where the code is currently.
<wgrant> The bug doesn't exist TTBOMK.
<barry> thumper: call me when you're ready
<barry> thumper: see what i was saying? :)
<thumper> barry: :)
#launchpad-dev 2009-08-13
 * rockstar goes to search for dinner
<wgrant> The project owner has an implicit subscription to all of the project's bugs unless there is a bug supervisor set. The bug supervisor can unsubscribe from its subscription, but the owner cannot. Is this intentional?
<wgrant> So, basically, you must have a bug supervisor set or the owning team gets spammed.
<wgrant> I wonder if it's a holdover from when bug supervisors couldn't unsubscribe either.
<maxb> Total: 23204 tests, 8 failures, 7 errors in 236 minutes 9.812 seconds.
<maxb> karmic, python2.4
<maxb> devel r9098
<wgrant> maxb: pastebin beckons.
 * kfogel is away: Smithwicks w/ Omar
<wgrant> I must set LP up in a karmic chroot...
 * thumper wanders up the road for a coffee
<maxb> Hmm, so several appear to be because I was holding out making sqlite available to python 2.4 until I verified what wanted it
<maxb> canonical/lazr/doc/timeout.txt failure... http://paste.ubuntu.com/252217/ - mean anything to anyone?
<maxb> test_realiseUpload_for_delayed_copies - http://paste.ubuntu.com/252218/ - DONE != ACCEPTED assertion failure
<wgrant> maxb: There were no errors before that last one?
<maxb> no
<maxb> and then lastly soyuz/doc/gina.txt and soyuz/doc/package-diff.txt are very unhappy
<wgrant> gina.txt gets unhappy sometimes.
<wgrant> ubuntu-meta sometimes fails to unpack on Karmic.
<wgrant> Even though it always works when I do it manually.
<wgrant> I might look further into that today, if it's that close to working on 2.4 Karmic.
<maxb> yup, confirmed, with the appropriate additional sqlite packages, down to just the 4 mentioned failures
<maxb> documented on LaunchpadOnKarmic
<wgrant> maxb: I can only reproduce the package-diff.txt and gina.txt failures.
<wgrant> The package-diff.txt one was an easy fix (debdiff in < Karmic is buggy, and doesn't obey the manpage wrt. exit codes; Karmic's does)
<wgrant> gina.txt I'm looking at now...
<wgrant> But it's non-deterministic :(
 * wgrant kicks gina a few times for being non-Zopey, non-Stormy, and non-deterministic.
<cprov> wgrant: on the bright side it is still importing debian 2x day with no complaints, has its merits.
 * cprov has to setup an karmic chroot, it's about time.
<wgrant> cprov: Indeed. It works mostly.
<wgrant> But has strange strange strange issues in karmic.
<cprov> maxb: on the test_realiseUpload_for_delayed_copies failure, print the 'logger.buffer' it might contain a hint about what's going worng.
<cprov> wgrant: paste me the complete failure if you can.
<wgrant> cprov: It's an error 29 (ESPIPE!?) running dpkg-source on a variable number of packages.
<cprov> wgrant: did anything sensible changed in karmic dpkg ?
<wgrant> cprov: I'm trying to work out exactly what it does so I can try to reproduce.
<wgrant> Since the command it says errored works fine.
<wgrant> So it might be something in Python.
<cprov> wgrant: `dpkg -sn` ?? there is no -n switch
<wgrant> cprov: Maybe not, but it works.
<wgrant> Just sometimes not when gina does it.
<wgrant> The number of failures differs each time.
<wgrant> Sometimes it succeeds.
<wgrant> cprov: Ah, -sn is one option. Of course.
 * wgrant waits for slow slow uni proxies.
<cprov> wgrant: yes, -sn exists, my bad
 * cprov goes to bed.
<wgrant> Night cprov.
<kfogel> barry: are you actually on right now?
<wgrant> (I noticed that it actually means the exit code was 29, not that an error with errno 29 occurred)
<kfogel> thumper: thanks for the review.  I'm getting 'bzr: ERROR: unknown command "pqm-submit" '... isn't pqm-submit part of the launchpad plugin?
<thumper> kfogel: no, it is bzr-pqm
<kfogel> thumper: ah
<kfogel> thumper: so I'm confused because it's *always* Just Worked from this box, and now suddenly it doesn't.  AFAIK I didn't do anything to change things... well, whatever.  I'll go look for the plugin.
<thumper> kfogel: which bzr?
<kfogel> thumper: trunk, recent
<kfogel> /usr/local/bin/bzr
<kfogel> symlink to my src
<kfogel> thumper: submitted
<thumper> spm: ping
<spm> thumper: heyo
<thumper> spm: can I get logs for the scripts bmp creation jobs
<thumper> spm: (both of them)
<thumper> spm: logs synced that is
<thumper> spm: I should see why I didn't get an email from karl
<spm> kk
<spm> thumper: bzrsyncd is done; doing codehost atm (can't recall offhand which one that stuff runs on)
<thumper> spm: me neither
<thumper> I'd love to rename the jobs so I can get it right
<spm> ignorance shared is.... hmmm. ignorance squared?
<spm> thumper: mpcreationjobs is bzrsyncd, fwiw
<wgrant> cprov, maxb: The gina.txt failure is a Karmic dpkg bug, which is probably maybe sort of tracked in bug #399938.
<mup> Bug #399938: unpacking the upstream tarball not working <apport-bug> <i386> <bzr-builddeb (Ubuntu):Fix Released> <https://launchpad.net/bugs/399938>
<wgrant> Hrm, the explanation in the bug was not clear.
<wgrant> It looks like it might actually be that the fixed dpkg bug has the same symptoms that are caused by Python ignoring SIGPIPE.
 * wgrant pokes harder.
<wgrant> There we go.
<wgrant> Fixed.
<thumper> spm: still cherrypicking?
<spm> thumper: ha. 4 hour test run? yes. :-)
<thumper> :(
<thumper> spm: plz get faster 'puter
<spm> thumper: pls write faster tests
<spm> last word. I win. :-P
<wgrant> I can haz ec2test and review?
<wgrant> lp:~wgrant/launchpad/karmic-2.4-fixes
<wgrant> I've only tested on karmic and jaunty, not hardy.
<thumper> spm: arg
<spm> thumper: hopefully should be finishing sometime in the next hour, hour & Â½.
<thumper> ok
<jtv> thumper: while you wait, can you help me figure out a problem I'm looking at?
<thumper> jtv: me waiting?  hah, but sure - hit me
<jtv> that thumper... last time he was in a slapping mood, now he's in a be-slapped mood
 * jtv hits thumper
<jtv> *any-way*...
<jtv> My script commits translations snapshots to a bzr branch.  Works fine generally.
<thumper> I feel a but coming along
<jtv> But there's a project that exports its translations back to the same branch it also imports translations from.
<jtv> That _should_ work.  The imports are based on branch jobs when the branch changes, the exports are done from a cron job.
<jtv> If the exporter sees that there's an unfinished translations import job on the branch, it backs off from committing to it.
<jtv> But I notice that for several days in a row now, every daily run sees this situation and backs off.
<thumper> ok...
<jtv> On staging there are such jobs, except they're all completed.
<thumper> how do you determine an unfinished translation import job
<jtv> Exactly.
<thumper> ah
<thumper> don't we have a start time?
<jtv> Well I look at type and status.
<thumper> OK, the way I see it you need two things:
<thumper> firstly
<lifeless> thing
<lifeless> ...
<lifeless> profit!
<jtv> lifeless: thanks, that's very helpful
<jtv> can we skip straight to the profit?
<thumper> unfinished should mean pending, that has started but not yet finished :)
<lifeless> jtv: I aim to please.
<thumper> rather than just pending in the future some time
<lifeless> better than aiming at peas.
<jtv> cringe
<thumper> and the import job would need to know not to run while an export is running on the same branch
<thumper> lifeless: better than aiming your pee
<jtv> no, actually aiming your pee is quite useful.
<lifeless> thumper: if you don't aim you will dis please
<thumper> jtv: as long as it isn't at you or me
<jtv> thumper: actually it's the other way aroundâI want the import job to complete peacefully and the export job to back off.
<thumper> it all depends on the target
<jtv> I'm just trying to figure out why the exporter hits that situation quite so often.
<thumper> jtv: yes, but if there is an export job running, you don't want the import job to start right?
<thumper> jtv: my guess is that when an import job has finished, it creates another at some future time
<jtv> thumper: the exporter has the branch locked anyway.
<jtv> I don't suppose the stuff I'm doing with the branch causes RosettaUploadJobs to be created even before I commit?
<jtv> AFAICT the import job does not create a new instance of itself: it's created by the branch scanner when a change is detected.  Besides, on staging I only see completed ones.
<thumper> hmm..
<thumper> I'd say get a losa to run queries for you
<jtv> it _might_ be an unlucky interference between cron jobs I guess
<thumper> to see if production is like you think it should be
<thumper> jtv: possibly
<thumper> but I don't think so
<jtv> owww, do I _have_ to talk to spm again?  :-)
<thumper> it seems wrong
<thumper> jtv: no, you could wait for herb :)
<jtv> spm it is then :)
<jtv> For the "I'm unwittingly creating the job myself" hypothesis I could ask a losa to give the export job an extra run...
<jtv> thumper: ahhhhh I think I may have it
<jtv> Job.id == BranchJob.id
<jtv> Bad join condition.
<thumper> haha
<thumper> yes
<jtv> Should be Job.id == BranchJob.job
<thumper> right
<thumper> jtv: I CC'ed you on my email about query performance
<thumper> jtv: I thought you might have some ideas
<thumper> jtv: it went to the launchpad-dev list anyway
<jtv> thumper: I see it
<jtv> and I owe you a look.  :)
<thumper> jtv: I _think_ we may be able to speed up the queries with an index
<thumper> jtv: I'm hoping
<thumper> or we may need some other magic
<jtv> thumper: I'll have a look.  Most well-documented "query performance post" I've seen so far.
<thumper> jtv: thanks
<jtv> thumper: this is a common pattern... IIRC BjornT had a similar optimization problem dealing with "either non-private or visible to this user."
<jtv> Maybe we should just have a team for Everyone.  :)
<jtv> thumper: you're always checking for ownership, even though you've got a done deal in the common case where the branch is public.
<jtv> thumper: (assuming the "private = %s" means "private is false")
<thumper> jtv: it is
<thumper> in the subscription one we only look for private
<jtv> you'll want to limit the ownership check to private branches
<thumper> do you think that it makes that much of a difference?
<thumper> perhaps it does
<jtv> thumper: it's hard to say; it's right at the other end of the join so maybe not.  But even then we'd hate ourselves for not checking.
<thumper> jtv: it could be huge
<thumper> jtv: given that we have 150k branchesnow
<jtv> child's play
<thumper> it is a one line fix
<jtv> wanna play with the TranslationMessage table?
<thumper> that should break no tests
<thumper> jtv: looked at the branch revision table?
<thumper> I think we win
<jtv> ah now, branch _revision_ table is big I know
<jtv> how big is it?
<thumper> biggish
<jtv> how biggish?
<spm> jtv: heyo
<thumper> spm: want to compare some timings for me?
<jtv> hi spm!  nm, thumper sat by the couch as I wrestled through my emotional problems and resolved the issue
<spm> jtv: errr. oki. I'll try not to feel left out, unwanted etc
<jtv> (It was something to do with my mother not loving me, or a bad join condition)
<spm> thumper: is that a trick question?
<thumper> spm: what is my person id?
<thumper> spm: I'm assembling the query now
<jtv> thumper: you can also move the selection of all public branches out of the union: "Branch.private = FALSE OR Branch.id IN (... UNION ...)"
<spm> thumper: 433852
<thumper> spm: http://pastebin.ubuntu.com/252311/
<thumper> spm: there are three queries there
<spm> thumper: oki
<thumper> spm: much appreciated
<spm> thumper: in return. does this look spurious or serious to you? https://pastebin.canonical.com/21094/
<jtv> spm, thumper: also, try this with a partial index on private branches only.  Indexing just the "private" attribute may not be quite as effective.
<spm> can't find any mention of that failed test beyond that.
<thumper> spm: spurious I think
<thumper> jtv, spm: the partial index thing is probably best done on staging :)
<thumper> jtv: can you give spm an example of what you mean?
<spm> I wasn't planning on doing it on prod! :-)
<thumper> spm: I wouldn't think you would
<thumper> spm: are you running my three on staging?
<thumper> spm: staging is fine I think
<spm> thumper: prod, but can shift?
<thumper> spm: I just need comparitive results
<thumper> spm: prod is fine too
<thumper> I think the first one takes around 1200ms
<spm> thumper: coolio. do you need the output or, sure you've got it right?
<thumper> I don't need the output, just the times
 * spm ndos
<thumper> the output should be the same for all three
<thumper> if it isn't
<thumper> something is wrong
<thumper> spm: it is just a count of active reviews I'm doing
<thumper> spm: it won't be very big
<jtv> I'd try "CREATE UNIQUE INDEX foo ON Branch(id) WHERE private IS TRUE" and "CREATE INDEX bar ON Branch(owner) WHERE private IS TRUE" and then run the query with the "private IS TRUE" condition on the ownership check.
<jtv> One or the other might catch something.
 * thumper waits 
<spm> thumper: 1st was 2.2secs, then 1.7 to 1.9secs on later runs. result is count = 1.
<thumper> I'm EODing after this
<thumper> spm: ok
<thumper> spm: that matches what I'm seeing in the oopses
<spm> thumper: 2nd was 2.9 for first; then consistant 1.8's; again count == 1
<thumper> ok, not much in that one then
 * thumper taps his foot
<spm> thumper: wow. I think you want to use #3. 0.127secs from the get go
<thumper> w00t
<thumper> jtv: you tha man
<spm> 10x improvement is impressive
<thumper> I think I can submit that fix easily
<spm> +1 from me for it at any rate :-)
<jtv> I haven't exactly followed the channel...  what was it that did it?
 * thumper branches now
<thumper> jtv: the third query in the pastebin
<thumper> jtv: moving Branch.private = False out of the union
<jtv> thumper: cool, I think I've repaid you for the day then.  :)
<thumper> jtv: that you have, thanks
<jtv> my pleasure, sir
<thumper> ok...
<thumper> not as trivial as I hoped
<jtv> uh-oh
<thumper> but I think I can get it done tomorrow
<thumper> it is more how we handle visibility checks in the branch collection code
<thumper> I was hoping for a two line fix
<thumper> and it will be longer
<thumper> but still relatively simple I think
<thumper> but not for today
<thumper> I'll look at it tomorrow
 * thumper acts like a tree
<jtv> Oh, I think I've looked at that code once and the function calls to build up the query go pretty deep.
<jtv> What do trees do?  Grow?  Metabolize carbon dioxide?  Get hugged by hippies?
<spm> trees? cut down and turned into beautiful furniture? a la my lovely jarrah desk I built that I'm sitting at atm
<jtv> spm: we don't want thumper to be get cut down and turned into furniture, now do we?
<jtv> (*Beautiful* furniture could be hard, but let's generalize the principle)
<spm> ah. that'll teach me for reading approximately ONE line of scrollback and responding. Doh.
<jtv> No worries.  All makes for good entertainment.
<spm> LOSA Duties: operational sysadmin; general entertainer; etc
<jtv> spm: dance for us!
<jtv> (At this point I bet ara is wondering...)
<spm> jtv: I dance... on  the inside
<jtv> ooh, nice comeback
<jtv> so nice in fact that I'll refrain from making jokes like "inside of what?"
<spm> heh
<maxb> +  * lib/lp/soyuz/tests/../doc/gina.txt - tar broke dpkg. dpkg was fixed. python breaks the fix. wgrant has a fix.
<maxb> whoa, sounds hellish :-)
<wgrant> maxb: Nasty signal stuff.
<wgrant> But see lp:~wgrant/launchpad/karmic-2.4-fixes
<wgrant> All fixed.
<wgrant> Soyuz tests pass now.
<wgrant> That one took a little while to track down.
 * maxb wonders how on earth we get valid archives wherein tar doesn't consume all the compressed data
<wgrant> Who knows.
<wgrant> But it happens...
<wgrant> Often.
<wgrant> maxb: What arch did you run the tests on?
<wgrant> Because I have no idea where those other failures came from...
<maxb> amd64
<wgrant> Seems unlikely, but that's one difference.
<maxb> I'd try again, but I'm in the middle of a py2.5 testrun
<maxb> I need to make myself a separate lp-sourcedeps for 2.4 vs 2.5
<wgrant> I need a separate sourcedeps for amd64 vs. i386 :(
<wgrant> That's why all my instances are i386; my first one was.
<maxb> I'm thinking symlink download-cache, bzr checkout --lightweight sourcecode/*
<wgrant> Possibly.
 * wgrant heads home.
<adeuring> good morning
<mrevell> Morning
<wgrant> Now that Bugs people are around:
<wgrant> The project owner has an implicit subscription to all of the project's bugs unless there is a bug supervisor set. The bug supervisor can unsubscribe from its subscription, but the owner cannot. Is this intentional? I wonder if it's a holdover from when bug supervisors couldn't unsubscribe either.
<wgrant> This confused when I moved a project to Launchpad last week, and couldn't unsubscribe the team.
<intellectronica> wgrant: i'm not sure i understand. what were you expecting to be able to do?
<wgrant> intellectronica: The project owner always has an implicit subscription to a project's bugs if the project has no bug supervisor.
<wgrant> intellectronica: This subscription is impossible to remove.
<wgrant> It is the only one which is irremovable.
<wgrant> This seems strange.
<intellectronica> wgrant: well, it's not really a subscription. it's an "also notified"
<intellectronica> the point is that _someone_ needs to be notified, and project owner or bug supervisor make sense
<wgrant> intellectronica: It's a forced structural subscription.
<wgrant> intellectronica: The bug supervisor can easily unsubscribe.
<wgrant> As the bug supervisor is no longer implicitly notified; it just normally has a structural subscription.
<wgrant> The project owner *is* implicitly notified, without a subscription.
<wgrant> This is entirely unobvious.
<wgrant> 'Subscribe to bugmail' is the obvious place to look, but doesn't help.
<wgrant> So I checked the code, and indeed it is hardcoded.
<wgrant> (lp.bugs.model.bug:620)
<wgrant> intellectronica: So, it is quite easy to get into a condition where nobody gets email, but also quite hard for users to work out how to stop email.
<wgrant> And other bugtrackers don't by default email anybody about all new bugs.
<wgrant> So it wouldn't be entirely unthinkable to not have the owner notified by default.
<intellectronica> don't know, maybe you're right and we don't have to have any subscribers by default. haven't though about this enough. i still think that it would be bad if a project had noone notified
<wgrant> intellectronica: It sounds like a question that should be asked during the guided project creation.
<wgrant> That can then create a structural subscription if the creator desires.
<intellectronica> wgrant: i'm not sure it's worth asking, but it might make sense to simply create a structural subscription (which can then be removed)
<intellectronica> there's one serious implication of having no subscriptions, though, and that's private bugs
<wgrant> intellectronica: That does get slightly confusing when the ownership changes.
<wgrant> intellectronica: Private bugs do not care about indirect subscriptions, do they?
<wgrant> Don't they just subscribe either the security contact or the bug supervisor?
<wgrant> Or owner if either doesn't exist?
<intellectronica> they do, in the sense that indirect subscriptions are converted to direct ones when a bug becomes private
<wgrant> Right.
<wgrant> But I'm not sure why that's of concern.
<wgrant> Or do you mean that you'd expect the project team to be able to see them always?
<wgrant> I think bug privacy implications need to be much more obvious, which would alleviate this.
<intellectronica> because you can end up with orphan bugs. bugs which nobody will be able to see. this is risky in particular with security-related bugs
<intellectronica> i agree
<wgrant> Bugs that are filed as security bugs will always have somebody subscribed, though.
<wgrant> And maybe marking a bug as private should warn if only you'll be able to see it.
<wgrant> But if a structural subscription is created initially, the projects with nobody subscribed should be minimal. In my case the interested developers subscribed separately.
<wgrant> But there are managerish people in the team, and they probably didn't want notifications.
<mrevell> jtv: What do you reckon we should call these "policies"? Review policies?
<jtv> mrevell: access or permissions policies
<mrevell> jtv: Okay, let's stick with permissions policies as that's what we've had pretty much all along.
<jtv> mrevell: cool
<beuno> g'mornin
<jtv> beuno: buenos
<beuno> jtv, que tal?
<jtv> beuno: bien, bien... Â¿tu?
<jtv> oh, I can do better: tÃº
<jtv> danilos: cp requests are both there now... what else have I forgotten?  :)
<beuno> jtv, "vos"
<jtv> beuno: ahhh, you funny folks  :)
<beuno> :)
<jtv> The two countries whose Spanish is _really_ weird and different: Argentina and Spain
<beuno> we like being unique
<salgado> hey beuno!  any news about those breadcrumbs?  I'd like to see some more examples to make sure the new infrastructure can cope with them
<beuno> salgado, I will write them *now*
<salgado> beuno, great, thanks!
<beuno> salgado, Im putting examples for:  a bug, a branch, a merge proposal, a package, a bug in a package and a blueprint. Need anymore?
<salgado> beuno, how about https://translations.edge.launchpad.net/ubuntu/jaunty/+source/gfxboot-theme-ubuntu/+pots/bootloader/af/+translate ?
<beuno> salgado, will add that as well
<beuno> working on the wiki page
<beuno> gimmi a few more minutes
<salgado> sure thing!
<BjornT> sinzui: ping?
<sinzui> Hi BjornT
 * gmb => caffeine. BBIAB.
<BjornT> sinzui: hi. i'm looking at c.l.javascript/registry/milestoneoverlay.js
 * sinzui saw the bug
<BjornT> sinzui: when you show an error message in the overlay, why don't you use overlay.showError()?
<sinzui> BjornT: It did not exist when the widget was created
<BjornT> sinzui: ok. so it's ok if i change it to use showError()?
<sinzui> BjornT: I think rockstar added the feature after looking at competing implementations. I talked to him about it
<sinzui> BjornT: +1 to fix the widget. If you do not have time, I think we will get to it before 3.0
<BjornT> sinzui: cool. i'll do it, since i want to update it to use a new method i wrote, and the custom error handling got in the way.
<beuno> salgado, that was a tricky one
<beuno> salgado, danilos, https://wiki.canonical.com/Launchpad/UI/Navigation
<beuno> I did something weird there
<beuno> Ubuntu > Jaunty 9.04 > Afrikaans translations > gfxboot-theme-ubuntu > Translating bootloader
<beuno> instead of:
<beuno> Ubuntu > Jaunty 9.04 > gfxboot-theme-ubuntu > Afrikaans translations > Translating bootloader
<beuno> it was an arbitrary decision, to fix a long-standing bug for translators who can't get back to their language to translate more templates
<beuno> I will continue adding the remaning examples, but let me know what you think
<danilos> beuno: right, it's something we discussed before, and what we want to fix anyway
<danilos> beuno: we need that on product translations as well
<beuno> danilos, yeah, and the links don't make sense in the second option, even though the hierarchy may
<beuno> I'm not 100% sure this isn't too crazy, but I'm 65% sure
<danilos> beuno: yeah, we can easily move the links around as well
<beuno> danilos, will do one for product translations. Do you have a link handy?
<danilos> beuno: https://translations.launchpad.dev/evolution/trunk/+pots/evolution-2.2/es/+translate (I think, I made half of it up :))
<danilos> beuno: or, if you want something on edge...
<beuno> danilos, yes  :)
<danilos> beuno: https://translations.edge.launchpad.net/tangocms/trunk/+lang/es and any link from there
<beuno> danilos, thanks
<danilos> kfogel: flacoste is in a crapper :)
<matsubara> stub, herb, flacoste, rockstar, bigjools, henninge, sinzui, intellectronica: LP production meeting in 47 min in #launchpad-meeting
<intellectronica> thanks for the reminder, matsubara
<herb> matsubara: thanks
<salgado> sinzui, one of the tests failed because I forgot to re-add the link to '+branding' that was removed from the actions menu. should I leave it in the nav menu for now?
<sinzui> salgado: Can you add the link to details page.
<sinzui> salgado: change details page.
<sinzui> salgado: EdwinGrubbs: Beuno provided a good plan to allow us to move branding links into the page, so we only need to provide the link from the change details page for now.
<salgado> sinzui, at the bottom of the +edit page, using the related pages thing? or something else?
<sinzui> salgado: That is the immediate kind of fix (and we can hack the h2 and ul into template, no menu) , the other route is add a sentence that includes the link in extra-form slot
<salgado> sinzui, +edit has been converted to 3.0 in this branch of mine, and it already has a <tal:menu replace="structure view/@@+related-pages" />
<salgado> so, if there isn't anything preventing me from defining a menu to be used there, I think I'll just do it
<sinzui> +1
<beuno> kfogel, http://googletesting.blogspot.com/2009/07/call-for-attendance-google-test.html
<salgado> sinzui, https://pastebin.canonical.com/21116/ fixes most of the tests that failed.  only 2 more to go
<sinzui> salgado: +1
<sinzui> salgado: if the product-menu test failed, make sure you have the latest version...the lastest version is not bittle
<salgado> sinzui, I've removed portlet-milestones from project-index.pt, so now there's a test failing because there's no 'See all milestones' link in that page
<sinzui> We should add the link I think. I do not think there is any other way to get to that page.
<salgado> sinzui, any suggestions for where to put the link?  I don't see any obvious place for it in the current layout
<sinzui> At the bottom of projects portlet, where we list milestones
<beuno> sinzui, could you comment on bug 412971?
<mup> Bug #412971: Convert bug-edit.pt to 3.0 template <Launchpad Bugs:In Progress by deryck> <https://launchpad.net/bugs/412971>
<deryck> beuno, sinzui -- I'm adding notes/questions/etc. to that bug right now with questions I have so far.
<beuno> deryck, I commented
<beuno> thought I did
<beuno> but something... odd happened
<beuno> or even better, "can we make it an open team"
<deryck> beuno, ah the "no portlets" rule will solve half of my questions then :)
<deryck> beuno, I should have refreshed first :)
<beuno> deryck, perfect  :)
<deryck> beuno, some questions are still relevant though
<beuno> deryck, will look and answer
<beuno> salgado, finished with the breadcrumb detailing: https://wiki.canonical.com/Launchpad/UI/Navigation
<beuno> (I will move that to the public wiki soon)
<EdwinGrubbs> sinzui: the base-layout.pt moved the informational messages just above the maincontent div. Is there any reason it can't be moved back in? I have one test that fails, and I assume there will be others that we'll have to fix, because we usually use find_main_content() instead of importing BeautifulSoup directly.
<sinzui> EdwinGrubbs: I think I can move back. I did not intend to move it
<EdwinGrubbs> ok
<gary_poster> matsubara-afk: lemme know when you can give me some background for the librarian-gc failure.  no rush, just trying not to forget my part ;-)
<sinzui> salgado: I would like your thoughts about how to proceed from this: https://pastebin.canonical.com/21124/
<bigjools> sinzui: woohoo that did the trick thanks
<bigjools> I don't get rounded corners on the boxes, is that intentional now or am I missing some css?
<salgado> sinzui, sorry, was on the phone; looking now
<sinzui> bigjools: is your css uptodate? There was a typo in the css where the *border-radius properties uses % instead of px.
<bigjools> sinzui: I think so, the project page has rounded corners in .dev for me
<sinzui> bigjools: they should appear in the side for all .side .portlet classes
<sinzui> bigjools: then I wonder if some divs are missing class="portlet"
<bigjools> not that I can see
<bigjools> the yui-u is the outer div and the portlet is the inner div right?
<barry> sinzui: so ~person index should be a main_only, right?
<sinzui> barry: no way an objects index will be main_side in most cases. see the design that was sent out months ago
<barry> right, so main_only is the right thing to use
<sinzui> barry: no, main_side because it is a primary object's index page https://devpad.canonical.com/~curtis/LP_userdetail.png is not ready. I want it
<sinzui> barry: do not work on this. I want the team page to land first. It has layout changes that the user page should use.
<barry> yep, i know
<barry> i'm trying to get some basic grounding here.  okay, so ~person does have side actions
<bigjools> sinzui: I see the problem.  I was using Konqueror, it doesn't round the edges.
<sinzui> bigjools: It should...there is something missing from the style. Which version are you using...the latest uses webkit and we cover that case
<bigjools> sinzui: I am using the very latest in kde4.3
<sinzui>     -webkit-border-radius: 5px;
<bigjools> I see it
<salgado> sinzui, have you considered a new interface for objects that provide a registrant/registration-date, plus changing existing classes to provide that interface? I think that'd be the best way forward as it'd also get us on the right path for standardizing the names we use for this
<sinzui> salgado: So add properties or fix models to support this?
<salgado> sinzui, I'd add aliases to the existing properties and mark the existing ones as deprecated
<intellectronica> EdwinGrubbs: around?
<sinzui> salgado: okay. I will take that approach
<EdwinGrubbs> intellectronica: hi
<mrevell> catch you tomorrow guys
<sinzui> deryck[lunch]: I replied to the bug, sorry for the delay
<sinzui> deryck[lunch]: I can get you a diff of another edit page that was modified to show all the changes you might make to convert a modification page to ui 3.0
<sinzui> deryck[lunch]: I added an example diff of a full conversion of a modification page: https://dev.launchpad.net/VersionThreeDotO/UI/Conversion
<deryck> sinzui, thanks for all that.  *very* helpful work.
<gary_poster> abentley, rockstar, on #launchpad someone is having problems pulling.  They say it is neverending and then they get
<gary_poster> bzr: ERROR: http://bazaar.launchpad.net/%7Eturl/%2Bjunk/ircbot/.bzr/repository/packs/dd8ac238ab24bfcbbb8620946cf7d032.pack is redirected to https://launchpad.net
<gary_poster> do one of you have time to check that out?
<rockstar> gary_poster, abentley is on holiday this week. I'll figure chat with them.
<gary_poster> rockstar: ok thanks much
<deryck> sinzui, ping.
<sinzui> hi deryck
<deryck> sinzui, hi! With generic-edit.pt, there's no way to get messages/notifications back to the template, right?
<sinzui> deryck: those are supplied by base-layout.
<sinzui> deryck: EdwinGrubbs discovers a bug this morning. the notifications are in the wrong place
<sinzui> deryck: EdwinGrubbs may have a fix ready
<EdwinGrubbs> deryck: yes, I changed it in my branch, which I will put into review today.
<deryck> sinzui, ah, ok.  So it should work soon if I continue to use generic-edit.pt?  I have broken tests that test for confirmation messages is why I ask.
<deryck> EdwinGrubbs, awesome!  Good news then :)
<intellectronica> bigjools: bear in mind my branch is going to spend quite some time in pqm, it's lazr so it gets a full run. if you rather have it killed so that your testfix gets in quickly it's fine by me
<intellectronica> let me know if you do so that i can resubmit
<bigjools> intellectronica: ah bugger, might be a good idea to do that
<bigjools> or trunk will sit broken
<intellectronica> yeah
<bigjools> mthaddon: would you mind doing the honours please? ^
<mthaddon> bigjools: stop pqm, remove intellectronica's branch?
<bigjools> mthaddon: yeah please, so mine gets in quicker
<bigjools> and then tom can resubmit
<mthaddon> intellectronica: ok, feel free to resubmit
<intellectronica> mthaddon: cheers
<mthaddon> yah
<rockstar> How do I enable the storm tracer for make run?
<rockstar> EdwinGrubbs, ^^ Maybe you know?
<thumper> rockstar: skype
<rockstar> thumper, hi.
<bigjools> thanks for pointing that one out intellectronica
<intellectronica> np
<sinzui> beuno: is there a reason we do not want to show distribution custom icons in links?
<beuno> sinzui, none at all
<beuno> it's a bug
<sinzui> beuno: I seem to have fixed it. It was an accident.
<beuno> sinzui, thank you
<maxb> ajax bug commenting is entirely shiny :-)
<EdwinGrubbs> rockstar: do you still need info on the storm tracer?
<EdwinGrubbs> sinzui: is there a bug open already for the team edit pages?
<sinzui> No
<sinzui> EdwinGrubbs: maybe
 * sinzui looks at EdwinGrubbs assigned bugs
<EdwinGrubbs> huh, how did I miss that?
<sinzui> EdwinGrubbs: You have two bugs. One for the index and one for the edit pages
<rockstar> EdwinGrubbs, I think we're okay now, thanks.
<maxb> uhoh. ajax bug commenting is shiny, but not when it fails to actually save your comment despite looking as if it did!
<maxb> oh
<maxb> no, it did save, but there was a quirky browser caching issue
<beuno> sinzui, https://staging.launchpad.net/bzr
<sinzui> Those file names are going to be difficult. What part can we hide? Maybe we need a smaller font
<sinzui> beuno: I think we need a santise the packages portlet. Show the the latest pacckage, or limit it to 3
<beuno> sinzui, exactly
<beuno> I think the length is not a problem, but the large amounts of them is
<EdwinGrubbs> sinzui: mp sent
<sinzui> beuno: the file names are a font-size larger than the body text
<sinzui> EdwinGrubbs: thanks!
<beuno> sinzui, ah, indeed they are
<Fly-Man-> Evening all
<Fly-Man-> I have succesfully installed the LaunchPad on a local server
<Fly-Man-> and made some Wiki changes on how to get it running so you can work it on a internal network
<Fly-Man-> but I have a Q about the source importer
<Fly-Man-> does it work in the development branch /
<maxb> I'm fairly sure that's illegal unless you rebrand it
<maxb> I wish it wasn't though, I'd quite like to use Malone myself
<Fly-Man-> okay, so the code importer doesn't work in the development branch ?
<maxb> I have no idea
<Fly-Man-> because waiting for 1 hour now and I see no change on the pages
<Fly-Man-> maxb: https://dev.launchpad.net/Running/LocalNetwork
<Fly-Man-> For getting it to run on a local network
<maxb> I'd imagine it probably needs to be kicked off from a cronjob
<maxb> Whilst it's nice that Canonical have open-sourced LP at all, it seems that they've diligently avoided distributing a configuration capable of running a local instance for production purposes.
<Fly-Man-> Hmm, but so far it's working like a charm
<Fly-Man-> yeah, and still need to find out how i can delete the "test" stuff and projects
<Fly-Man-> so I can have a clean tree
<maxb> Fly-Man-: What do you mean by code importer, ooi?
<Fly-Man-> maxb: the part where I put in a SVN url
<Fly-Man-> and it starts to grab that SVN
<Fly-Man-> and downloads the SVN history into the trunk on the Launchpad
<Fly-Man-> so I can fiddle with it
<Fly-Man-> and checkout the source with bzr
<Fly-Man-> Like it tells on this page:
<Fly-Man-> https://help.launchpad.net/VcsImports
<maxb> It probably has something to do with cronscripts/code-import-dispatcher.py
<Fly-Man-> And also what I notices
<Fly-Man-> the version of the trunk build is still 2.2.6
<maxb> huh, still? I though wgrant landed a branch to fix that
<Fly-Man-> I see this:
<Fly-Man->  Launchpad 2.2.6 (r9108)  devmode
<Fly-Man-> I grabbed the source 4 hours ago
<maxb> hmm, wonder what happened to that branch
<Fly-Man-> no clue ...
<thumper> Fly-Man-: yes the importer stuff works
<Fly-Man-> thumper: is there any page on the Wiki about it ?
<Fly-Man-> how to setup ?
<thumper> no
<thumper> and right now I'm firefighting
 * Fly-Man- throws a fire blanket to thumper 
<Fly-Man-> thumper: hope it's not your pc that's on fire ....
<thumper> flacoste: no, production issues
<Fly-Man-> okay ...
<thumper> flacoste: sorry, tab completion fail
<Fly-Man-> well, i'll have some sleep
<Fly-Man-> and maybe tomorrow when it's not fire time we can have a talk ?
<thumper> it is friday for me today :)
<Fly-Man-> thumper: it's almost friday for me here
 * Fly-Man- yawns
<Fly-Man-> K, time for me to get some sleep
<Fly-Man-> Nite all :)
<Fly-Man-> and thanks for the help so far
<wgrant> maxb: Mine was r9109
<wgrant> Landed 12:41 UTC
<maxb> not that branch, different branch - what happened to your "fix the version numbers to say 2.2.7" branch?
<wgrant> Oh.
<wgrant> I didn't have a branch to fix that.
<maxb> ah. someone did. I must be misremembering
<wgrant> I tried to run the test suite in Karmic 2.4 last night, but my LVM snapshot filled up :(
<intellectronica> wow, something very strange is happening. i've submitted a lazr.restful branch to pqm and pqm spat it back claiming there are merge conflicts. but there are no revisions in trunk that are not present in my branch!
<thumper> intellectronica: crisscross merges?
<intellectronica> thumper: no no, my branch is exactly one revision on top of what's already in trunk. should be a very straightforward merge
<thumper> intellectronica: are you sure you are submitting to the correct branch?
<intellectronica> yeah, i'm wondering that too. i'm submitting to ~launchpad-pqm/lazr.restful/trunk
<intellectronica> which is the branch i branched from, so it is, actually, the correct branch
<wgrant> intellectronica: That's not really trunk.
<intellectronica> wgrant: oh?
<wgrant> Recent commits have been to ~lazr-developers/lazr.restful/trunk
<wgrant> Not via PQM either, AFAICT.
<intellectronica> which doesn't conflict with my branch either
<intellectronica> i find it very hard to believe that whatever we're using in lp accept merges not through PQM
<intellectronica> maybe it's a parallel branch or something
<intellectronica> you're right, though, that this is the branch currently registered as trunk
<intellectronica> leonardr: any idea what's going on? ^^^^^
<intellectronica> trying to submit to that branch fails too, which sort of makes sense, because it isn't owned by pqm
<intellectronica> ha ha, ok, i get it
<intellectronica> wgrant: thanks, your clue helped
<intellectronica> so my branch doesn't conflict with the branch i thought is the one i'm submitting too, which is the lazr-developers branch, but it conflicts all over the place with the branch managed by pqm
<wgrant> intellectronica: Excelent.
<wgrant> Ah.
<wgrant> Makes sense.
<wgrant> Apart from why the branch isn't PQMed.
<intellectronica> yeah, that's very strange, i don't remember any mention of that change
<wgrant> I've seen a couple of mentions of moving to a non-PQMed trunk in lazr.restful merge proposals as they fly past.
<maxb> What's the bin/test incantation to run a single doctest?
<wgrant> bin/test -vvt testname.txt
<wgrant> That's what I use.
<wgrant> You can also give the full path.
<maxb> wgrant: many thanks for sorting gina.txt. Unfortunately package-diff.txt is still broken.
<maxb> http://paste.ubuntu.com/252805/
<wgrant> maxb: Huh.
<wgrant> My Karmic LP chroot is currently rebuilding, so I'll look in a moment.
<wgrant> But that's nothing like the failures I had yesterday.
 * maxb rants furiously at shhh.py
#launchpad-dev 2009-08-14
<wgrant> maxb: OK, now I'll run that test..
<wgrant> Let's see how it goes...
<wgrant> maxb: Runs fine here.
<wgrant> I'm confused.
<wgrant> You have a few other tests failing that work for me, don't you?
<maxb> one other - LaunchpadOnKarmic is up to date
<wgrant> maxb: Ah, hadn't seen that latest change.
<maxb> right, let's see if it's arch-specific
 * maxb runs it on netbook....
<wgrant> I would try on amd64, but that means cleaning out sourcedeps which I don't have a good process for yet.
<maxb> rm -rf eggs/* && for d in sourcecode/*/; do (cd $d && bzr clean-tree --force --ignored --unknown --detritus); done
<wgrant> I suppose so!
<maxb> I guess if you're lucky you can leave the eggs
<wgrant> They should be version/arch-specific
<wgrant> So yes.
<wgrant> We'll see.
 * maxb has been caught out by py2.5/UCS4 vs. py2.5/UCS2
<wgrant> Hm.
<maxb> I'd forgotten about a Python I had in /usr/local/bin/
<wgrant> Somebody broke Python 2.6 like that in late Jaunty, I recall.
<wgrant> Caused a lot of things to die in strange ways.
<wgrant> Fortunately most had missing symbols, so it was obvious.
<maxb> Right, well I've reproduced the DONE != ACCEPTED one on entirely separate Karmic installations on different hardware, one amd64, one i386, so I've no idea how it's passing for you! :-)
<wgrant> Intriguing.
<maxb> package-diff.txt is grinding away on the netbook now
<wgrant> I might try cleaning out my ~/launchpad and starting from scratch.
<wgrant> Overkill perhaps, but I can't see anything else wrong :/
<maxb> package-diff.txt failure likewise reproduced
<wgrant> Damnit.
<weled> Hi, how to manage update ? rocketfuel-get and what ?
<wgrant> Would a branch to convince rocketfuel-setup to use bzr+ssh for the initial download be accepted?
<wgrant> It still forces http.
<maxb> Isn't that deliberate?
<wgrant> It was.
<wgrant> Not sure there's any reason for it to be now.
<wgrant> spm: Is LP still
<wgrant> ... still hot?
<wgrant> There was server-side support to force HTTP, which has probably been turned off now.
<wgrant> So I don't know why it's forced in the client too.
<maxb> how did the server know?
<wgrant> There's a config option which specifies products whose development focus branches should only resolve to HTTP, not bzr+ssh.
<wgrant> Added just before the release.
<intellectronica> sinzui: maybe you know what's the new deal with lazr branches?
<spm> wgrant: I may be missing the thrust of the discussion, but you could always - aiui - branch using http. It's a much lighter process at the cost of having the branch parent be not so helpful
<thumper> lp is not hot any more :)
<thumper> wgrant: we could update it
<wgrant> spm: And it daaaaaaaamn slow.
<lifeless> wgrant: http is used when you haven't done lp-login
<wgrant> lifeless: I know.
<wgrant> But rocketfuel-setup explicitly uses an HTTP URL to lessen the load.
<wgrant> Which probably isn't a valid concern any more.
<lifeless> wgrant: if the rocketfuel-get script either overrides HOME, BZR_HOME, or that setting specifically, ...
<lifeless> oh, it does what?!
<lifeless> insane
<lifeless> in the membrane
<lifeless> I keep saying this; there is no good evidence that the load is lighter over http
<lifeless> and some serious reason to believe that its /a lot higher/ at the moment.
<lifeless> because we have bugs.
<wgrant> 2a over HTTP really really sucks.
<lifeless> yes
<wgrant> Although it's not much better over bzr+ssh at the moment.
<lifeless> it makes millions of requests rather than 5
<wgrant> (using latest nightly on the client, and LP as the server)
 * thumper doesn't use rocketfuel-* scripts
 * wgrant only uses rocketfuel-setup
<lifeless> wgrant: interesting. it should still be streaming
<wgrant> lifeless: It is.
<wgrant> But at ~100kbps
<wgrant> Oops. KB/s.
<wgrant> But still very very slow, even from the UK.
<thumper> ha crap
<thumper> there is a failure on the devel builder
<thumper> soyuz bindarypackagerelease-views.txt
<lifeless> wgrant: whats your link capable of?
<lifeless> and when did you move to the UK ? :P
<wgrant> thumper: There was a testfix for that one this morning, wasn't there?
<thumper> was there?
<thumper> wgrant: yes there was
<wgrant> lifeless: 10Mbps. I can nearly saturate that from a UK host normally.
<thumper> thanks for the heads up
<wgrant> thumper: It was the same one? (I can't see buildbot)
<thumper> wgrant: it seems so
<lifeless> wgrant: have you alerted sysadmins? bzr should saturate the link.
<lifeless> spm: ^
<thumper> lifeless: how do we test where the blockage is?
<wgrant> lifeless: Let me test a host just near LP to see how the bandwidth is now...
<elmo> what are youguys talking abou
<elmo> the rewrite script is fucked
<elmo> you know this
<wgrant> This is bzr+ssh, though.
<elmo> I know you know this
<lifeless> elmo: wgrant is saying that the bzr+ssh streaming performance is underpar
<elmo> oh
<elmo> sorry
<thumper> :)
<lifeless> thumper: I'd start by looking for swapping
<lifeless> thumper: failing that it might finally be a reproducible case of the problems the python VCS PEP guy had
 * lifeless starts a stream up
<thumper> lifeless: maybe
<wgrant> I'm getting 600KB/s from the UK now, so it does seem to be an LP problem.
<lifeless> I'm seeing 50-80 KB/sec - 40-160kbits, on a 1500kbit link
 * lifeless tries mysql
<elmo> can y'all try downloading a large file from  the librarian to rule out general network issues?
<wgrant> woah.
<lifeless> elmo: yup, I was just looking for one
<wgrant> bzr is eating around a gigabyte of RAM.
<elmo> http://launchpadlibrarian.net/28202431/dia_0.96.1-7.1_0.97-2.diff.gz
<elmo> lifeless: ^-- is one I have in my browser history that isn't tiny
<jelmer> ~.
<wgrant> That sits at around 500KB/s for me.
<wgrant> Although it's not precisely large.
 * wgrant fetches a buildd chroot.
<lifeless> 3,730,692    101K/s   in 36s
<lifeless> elmo: ^
<wgrant> Hrm.
<wgrant> bzr is now streaming at ~350KB/s
<elmo> super, so not my problem ;-)
<wgrant> So much better, but still not as good as it could be.
<wgrant> ... and back below 100KB/s
<lifeless> elmo: did you just disown all of codehosting? :)
<lifeless> possibilities:
<lifeless>  - ssh windowing problem
<lifeless>  - bzr networking issue
<lifeless>  - bzr performance problem
<wgrant> bzr is eating hundreds of megabytes of RAM at the moment.
<wgrant> Even branching lp:launchpad locally.
<wgrant> Both 1.17 and bzr.dev.
<wgrant> How odd.
<elmo> lifeless: no, but there are other people available right now more qualified and suited to diagnose operational LP issues, as opposed to operational network issues
<lifeless> elmo: I was just teasing
<rockstar> I need claws to say something like "Hey, it looks like you're trying to do a code review in this email.  Did you remember to sign it?"
<lifeless> wgrant: is it still very slow for you?
<lifeless> wgrant: can you wireshark and see if the window is closing up (would indicate bzr not reading fast enough)
<lifeless> wgrant: who is your isp?
<sinzui> intellectronica: ping
<intellectronica> sinzui: hi
<sinzui> I'm going to reply to your email.
<sinzui> I think I know the answer. But I heard last week that we should not submit
<sinzui> intellectronica: If leonardr gave the all clear than you can
<intellectronica> what, push to the lazr-developers branch?
<sinzui> intellectronica: their two branches listed in my email, the pqm one that launchpad uses and the official lazr.restful, which you must prepare a patch for (mine did not apply cleanly)
<intellectronica> leonardr reviewed my branch, so presumably he's ok with that
<sinzui> intellectronica: lazr.restful. I presume so to
 * sinzui looks to both branches
<intellectronica> my branch is of the laze-developers one, so it applies cleanly. it doesn't apply cleanly against the pqm-managed one, but it shouldn't be hard to prepare a patch for that. it's a really simple change
<sinzui> okay
<intellectronica> the question is, how do it get my change in?
<intellectronica> should i submit to pqm, which is targeting an outdated branch, or somehow get it into the lazr-developers branch. and if the latter, how do i then get my changes used in LP?
<sinzui> for he pqm one, yes
<intellectronica> sinzui: you sure? it seems to me that what we're using in LP is the newer branch (from casual inspection of the code)
<sinzui> I am sure you need to submit to two places with different rules
<intellectronica> no, actually i may be wrong. maybe it is the pqm branch we're using
<sinzui> intellectronica: I need to put this in an email. I cannot think or type quickly at the moment
<sinzui> intellectronica: we are using the pqm. we then backport to our branch
<intellectronica> sinzui: you can probably imagine what state i'm in. i'll look at this again in the morning and i'm sure everything we'll be clearer :)
<sinzui> okay. You will have an email which will help
<intellectronica> sinzui: thanks a lot!
<lifeless> wgrant: ping
<lifeless> wgrant: I suspect you weren't using bzr+ssh
<wgrant> lifeless: Sorry, was in transit to work.
<lifeless> 9684KB   153KB/s | Fetching revisions:Inserting stream
<wgrant> It certainly was.
<lifeless> lp: appears to still use http
<wgrant> ISP is Optus (Cable)
<wgrant> Right.
<wgrant> But I gave it bzr+ssh explicitly.
<lifeless> bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel ?
<wgrant> Precisely.
<lifeless> ok, well thats good to know
<wgrant> bzr.log agrees that it was using that.
<lifeless> owever, I see my adsl saturated
<lifeless> which is good
<lifeless> wgrant: if you can reproduce, please be getting tcpdumps
<lifeless> and filing bugs
<lifeless> if its a bzr bug, we want to fix it
<lifeless> if its an operational issue we want to fix it
<thumper> I'm going to be around but not near my computer for a while while I help with a submission to stop a huge building go up across the road
<thumper> rockstar: if you need me, text :)
<lifeless> thumper: when you get back
<lifeless> is lp: meant to still force http? if its not, then something isn't deployed.
<thumper> lifeless: I'm not back yet, but the change has landed but hasn't been rolled out
<wgrant> lifeless: I think it's probably a general bzr problem, not specific to streaming. It's really slow and memory hungry even doing a local branch. Can you try to reproduce?
<wgrant> (branching 2a lp:launchpad from a local shared repo to a new branch outside the repo.
<lifeless> wgrant: do you have the C extensions?
<wgrant> lifeless: The only .so I can see is _patiencediff_c.so. This is the ~bzr-daily-ppa build.
<lifeless> then you don't
<lifeless> check in bzrlib/bzrlib/
<lifeless> james_w: around?
<wgrant> lifeless: Let's try Karmic's 1.18rc1, which might be more sane...
<wgrant> lifeless: That's muuuuuch better locally.
<lifeless> file a bug please
<lifeless> assign to james_w
<lifeless> on bzr
<wgrant> Although it's still eating lots of RAM and CPU, it's much faster.
<wgrant> Will do.
<wgrant> Thanks.
<lifeless> its caused by the bug I've filed about builds on karmic doing WEIRD SHIT with were the .so's go
<lifeless> but the daily debs should seperately be erroring/requiring the .so's
 * wgrant disappears for lunch.
<jtv> spm, g'f'day mate!
<spm> jtv: hey!
<jtv> an Australian dared me to pinpoint his accent last night, and I couldn't even say for sure whether he was British or not.
<spm> :-)
<jtv> In all fairness, he was from Adelaide, born in London, from two English parents.  It all happened because I caught a few words his friend said and went "he's from Manchester, isn't he?"
<jtv> (Apparently he lived in London and only betrayed bits of his original accent given sufficient beer)
<jtv> spm: but seriously, I've got two CP requests up for the codehosting server... any chance we can do them nowish?
<spm> jtv: funny you should ask...
 * jtv cowers behind laptop
<spm> both (plus 2 others) have passed testing
<jtv> no wait, this thing is expensive
 * jtv cowers in front of laptop
<jtv> ...was there a "but" coming?
<spm> I was in the process of re-cowbowing the changes preparatory to pushing when a ... bigger priority fail hit
<spm>  -ETOOMANYFAILURES
<spm> that seems to have recovered sufficiently atm, that I should be able to get back on the CP in <5
<jtv> catch (const exception &e) { puts(e.what()); }
<spm> heh
<jtv> (I mean, "what happened, Steve?")
<thumper> puts?
<thumper> ew
<thumper> std::cerr << e.what() surely
<thumper> and it should be "catch (exception const& e)"
<thumper> ask me why some time
<jtv> thumper: that's ugly... like "1 == x" because some people _might_ occasionally write assignment instead of comparison.
<lifeless> thumper: the whitespace for the &?
<jtv> btw if you're going to std::cerr << e.what() then don't forget the eol.
<thumper> << '\n'
<thumper> const& on the RHS of the type
<jtv> thumper: now now, who's being all old-school a-better-C here?
<thumper> cerr flushes automatically, so explicit not needed
<jtv> thumper: the const and the & have basically nothing to do with each other... the const qualifies the exception, not the &
<jtv> I know clog flushes automatically, but cerr..?
<thumper> jtv: we need beer and time to talk this through
<wgrant> Now now people. Let's keep the channel Python-friendly.
<thumper> cerr yes
<jtv> thumper: aye to that!
<thumper> wgrant: why?
<wgrant> thumper: C++ is too much like Java.
 * jtv screams
<jtv> We were here first.
 * wgrant kicks Python 2.4 to death.
<jtv> wgrant: "don't kill it yet, we still need it"
<thumper> spm: are you un-paniced yet?
<spm> nope
<thumper> I'll go away and check back later...
<lifeless> thumper: oh doh, I should have noticed the const positioning
<lifeless> of course, its a PITA.
<wgrant> maxb: Running 'make check' on karmic amd64 with 2.4 has so far given me a LayerIsolationError in timeout.txt, but package-diff.txt just passed fine. I think LP might hate you.
<wgrant> maxb: This is all freshly downloaded and installed.
<thumper> I want pizza
 * thumper is thinking of Poppa's pizza
<lifeless> mmmmm
<lifeless> do not be tempting me like that
<thumper> spm: reminder to re-enable pqm :)
<spm> oo. ta. will do.
<wgrant> Why must PQM be turned off for CPs?
<lifeless> the same machine is used to validate the production branch changes
<lifeless> wgrant: e.g. 'why can't you run two lp test suites at once on the same machine'
<wgrant> lifeless: Ah. I thought buildbot was all EC2, so the CP tests would be on another instance.
<wgrant> But that makes sense.
<poolie> spm: are you working on this now?
<spm> poolie: bzr access? very much so.
<spm> the CP went sour in a big way
<poolie> cp?
<spm> cherry pick
<wgrant> Ooh dear.
<spm> we've reverted - so back to where we were; but still open to the HTTP errors. :-/
<poolie> i actually really like sour cherries
<wgrant> Was it the rewrite stuff that was CPed?
<spm> poolie: a sincere thank you! jokes during a crisis are *always* appreciated :-)
<wgrant> poolie: They are good, yes.
<spm> wgrant: amonst some other bits, yes.
<poolie> spm, ssh seems to still be down
<spm> poolie: yes; trying a hopefully fixed update. should be back now.
<adeuring> good morning
<jtv> hi adeuring!
<bigjools> morning
<gmb> Morning.
<jtv> hi bigjools, hi gmb
<wgrant> Why is my postgres eating a couple of gigabytes of disk after a test run failed due to a lack of disk space?
<wgrant> Is it told not to clean up, or something?
<bigjools> autovacuum not running?
<wgrant> I was thinking that.
<wgrant> But I don't see why it wouldn't be.
 * wgrant checks more carefully this time.
<gmb> I've had it vanish silently on me before.
<gmb> But it's been a while since that happened (might even have been back when we were on PG8.2 rather than 8.3)
<wgrant> I shall watch more carefully this time.
<wgrant> This has probably happened all five times I've run 'make check' now, but the first three times I didn't realise it was the database.
<wgrant> So it seems unlikely that autovacuum is going missing.
<mrevell> Morning
<deryck> Morning, all.
<Fly-Man-> morning thumper
<mrevell> hey Fly-Man-, thumper's most likely afk ... it's late Friday evening for him
<Fly-Man-> mrevell: Ahh, he's the Australian one ?
<Fly-Man-> mrevell: morning btw :)
<mrevell> Fly-Man-: Heh, New Zealand -- I think it's a capital offence to confuse the two. Morning :)
<Fly-Man-> sorry thumper ;)
<Fly-Man-> mrevell: I asked a question yesterday about the launchpad download
<Fly-Man-> as it seems the version is still the 2.2.6 version
<mrevell> Fly-Man-: The LP source code?
<Fly-Man-> and i'm part of the beta team
<mrevell> Fly-Man-: Ah, the version number that appears in the footer on edge?
<Fly-Man-> and it seems the beta site is running a lower version then my local launchpad
<Fly-Man-> which surprises me ...
<Fly-Man-> mrevell: Yes, the source that you can get
<Fly-Man-> My local version is
<Fly-Man-> Launchpad 2.2.6 (r9108)
<Fly-Man-> the version from edge is
<wgrant> edge only updates every 24 hours, and it updates from a branch that is always several hours out of date.
<wgrant> So yours is very probably newer.
<Fly-Man-> Launchpad 2.2.6 (r9041)
<Fly-Man-> wgrant: yes, but thumper also mentioned that the new 2.2.7 version should be available as download as well
<Fly-Man-> but I can't seem to find that one anywhere
<wgrant> Fly-Man-: There's no '2.2.7' version.
<wgrant> It's all the same branch.
<wgrant> But the version number hasn't been changed yet.
<mrevell> Fly-Man-: The version number that shows in the footer isn't all that reliable. Usually it is but it's actually just a text string that's manually updated.
<Fly-Man-> Ooo, so the version I download is the "2.2.7" ?
<mrevell> Fly-Man-: The version you download is whatever is the latest version
<mrevell> 2.2.7 only exists as a snapshot in time on the live production server
<Fly-Man-> Ahh, that makes perfect sense
<mrevell> So, the version you download is 2.2.7 plus whatever has been committed since the 2.2.7 release.
<Fly-Man-> Ahhh, that's good :)
<Fly-Man-> so I don't need to perform the rocketfuel-setup every time to get the latest version
<Fly-Man-> but can just pull the version with bzr
<Fly-Man-> but now a question that might be a hard one ....
<mrevell> Fly-Man-: Or run utilities/rocketfuel-get
<Fly-Man-> "How to get the code importer function to work"
<Fly-Man-> and clean up the projects that I don't need in there
<Fly-Man-> because thumper mentioned that the codeimporter should work
<Fly-Man-> but then had to fire fight something on the site
<wgrant> What exactly are you trying to do?
<Fly-Man-> I am trying to import the SVN trunk from a server
<Fly-Man-> to see if I get the same errors that the launchpad.net gets
<Fly-Man-> to analyse where it might go wrong
<Fly-Man-> and also planning on the Git import
<wgrant> All of the Code guys are hopefully asleep, so there's probably not much help around for that right now.
<mrevell> abentley should be around in two or three hours
<Fly-Man-> so I can match the problem and see where that might have a patch/fix that might help
<wgrant> Git is easy to test.
<wgrant> Just install bzr-git and try to check the repo out with it.
<wgrant> SVN is harder, because cscvs is old and confuses me.
<Fly-Man-> k, bzr-git doesn't come up when I do a apt-cache search bzr
<Fly-Man-> so I might not have the right repository's set
 * Fly-Man- does a lightbulb above his head
<Fly-Man-> See, I was right about 1 thing ...
<Fly-Man-> https://code.edge.launchpad.net/~vcs-imports/opensim/svn-trunk
<Fly-Man-> if I check it out with bzr svn-import url
<Fly-Man-> it gives an error
<Fly-Man-> when I check it out without the trunk
<Fly-Man-> it's importing ...
<wgrant> Fly-Man-: LP doesn't use bzr-svn.
<Fly-Man-> okay ...
<wgrant> It uses cscvs, from which you want to swiftly escape.
<Fly-Man-> So in other words, cscvs does the things different
<wgrant> Yes.
<wgrant> Completely.
<Fly-Man-> and a code change would be required to make it go from cscvs to bzr-svn /
<wgrant> Yes, but there are bigger problems with the transition.
<wgrant> I believe it is on the list for post-3.0
<Fly-Man-> I keep wondering why it gets those messages on the SVN
<Fly-Man-> WARNING N changeset 1
<Fly-Man-> WARNING N changeset 2
<Fly-Man-> and then it just "poofs"
<wgrant> gmb, bigjools: So, autovacuum is running as it should, and even manually vacuuming doesn't help :(
<gmb> Hrm.
<bigjools> o\
<wgrant> 'make schema' almost cuts it back to where it started. So I'm pretty confused.
<Fly-Man-> mrevell: I just found a simular bug like the one I have
<Fly-Man-> and it seems Paul Hummer is working on that
<Fly-Man-> Would it be wise to match the bug I made and put it on that bug report as well ?
<mrevell> Ah, that's good to hear -- he's on US mountain time so he'll be around in two or three hours
<mrevell> Fly-Man-: If you've already made a bug report, you can mark it as a duplicate of the one Paul's working on
<Fly-Man-> k
<Fly-Man-> wgrant: 2009-08-14 13:09:34 ERROR   No mail box is configured. Please see mailbox.txt for info on how to configure one.
 * Fly-Man- doesn't see a mailbox.txt file anywhere ?
<Fly-Man-> kfogel: Goodafternoon kfogel
<Fly-Man-> kfogel: You were gone when I was finished
<Fly-Man-> but here's the page to the local setup
<Fly-Man-> https://dev.launchpad.net/Running/LocalNetwork
<kfogel> Fly-Man-: reading now
<Fly-Man-> k
<kfogel> Fly-Man-: is this only for machines on your local network?  Would it work for machines on the WAN too?
<kfogel> Fly-Man-: IOW, I'm wondering if the name of this page should be "RemoteAccess" or something
<kfogel> Fly-Man-: the summary at the top says "This page tells you how to change the Launchpad so you can access it on your own machine."  I think maybe that's too vague?  After all, that's what the normal Running instructions give.  Your tutorial is about something bigger: the ability to access Launchpad from a *different* machine than the one it is running on (right?).
<Fly-Man-> kfogel: It will work on a Wan as well
<kfogel> Fly-Man-: so, let's rename the page.  It's not linked to from anywhere yet, right?
<Fly-Man-> But for a WAN I would suggest using a DNS server that an register
<Fly-Man-> it's only registered at the Running page
<kfogel> Fly-Man-: I'm sorry, I didn't understand your last sentence.
<kfogel> oh
<kfogel> linked
<Fly-Man-> yes
<kfogel> you used "register" twice, but with different meanings :-)
<Fly-Man-> sorry, Dutch ;)
<kfogel> heh
<kfogel> Fly-Man-: we can fix the Running link, no problem
<Fly-Man-> RemoteAccess could be a good name for that
<Fly-Man-> but then it needs a step with it for Wan usage
<kfogel> so, first: let's rename the page.  then, if it needs any tweaks to explain difference between LAN and WAN access, maybe you can put those in?
<Fly-Man-> "If you want to use it on a Wan, please make sure you can access a DNS server to setup the values"
<kfogel> Fly-Man-: I can rename right now and fix Running, if yuo wnt.
<Fly-Man-> go ahead :)
<kfogel> "you want", sorry
<kfogel> typo
<kfogel> ok
<kfogel> Fly-Man-: all done
<kfogel> Fly-Man-: fixed summary bar too
<Fly-Man-> k
<kfogel> Fly-Man-: I'll leave the WAN changes to you, since you know the material better
<Fly-Man-> Well, that's the easy part ;)
<Fly-Man-> kfogel: done ;)
<kfogel> Fly-Man-: thanks!
<Fly-Man-> Yw
<Fly-Man-> and now to find the people that can explain the parts that I miss
<Fly-Man-> like the setting up of the cron jobs
<Fly-Man-> I have seen that some jobs run normally
<kfogel> Fly-Man-: "When you have done this correctly, then you will be able to use the local launchpad application within your own WAN"
<Fly-Man-> and some need additional fixing
<kfogel> LOL -- I wish I had my own WAN! :-)
<Fly-Man-> kfogel: You don't ;) ?
<kfogel> Fly-Man-: mind if I tweak to "the WAN"?
<Fly-Man-> it's already tweaked ;)
<kfogel> Fly-Man-: see it now, how does this wording look?
<Fly-Man-> nice
<Fly-Man-> looks good to me
<kfogel> Fly-Man-: cool
<Fly-Man-> kfogel: But now the configuration part ...
<Fly-Man-> because I see MANY python scripts in the cron-scripts folder
<Fly-Man-> some do something
<Fly-Man-> but i found out the hard way, some are related to the launchpad.net version
<kfogel> Fly-Man-: I am about as knowledgeable as you are about those...
 * Fly-Man- smiles
<Fly-Man-> Aha, and I was thinking you're the Guru master ;)
<Fly-Man-> But who would be the ones to ask those Q's /
<Fly-Man-> so we can make the configuration of those index on the Wiki as well ?
<Fly-Man-> mrevell already told me maybe abentley  would be one of them
<Fly-Man-> but the best part is the SSL certificate that's invalid :|
<jmux> Hi. I've set up Launchpad and now want to use sync-source to sync the Debian sources into my launchpad? I've found the ArchiveAdministration wiki page, but this references an update-source script in a ~/syncs directory. Is the content of ~/syncs available somewhere?
<maxb> kfogel: Fly-Man-: A further comment on that page - IMO it's too wordy by far. I'd prefer to strip all tutorial-esque information out and turn it into a document on what you change
<Fly-Man-> maxb: Feel free to edit it :)
<maxb> Well, I'm always a bit hesitant about deleting other's content
<Fly-Man-> maxb: Go ahead :)
 * Fly-Man- approves to it
<james_w> jmux: it is 'run "update-sources" in ~/syncs', rather than 'run ~/syncs'
<kfogel> Fly-Man-: I'm not a Launchpad guru by any means.  Still learning.
<james_w> jmux: I'm not sure where update-sources comes from though
<Fly-Man-> kfogel: *grin*
<Fly-Man-> Same here
<Fly-Man-> but so far, I have gotten some experience
<maxb> Launchpad doesn't like me
<kfogel> maxb: ?
<Fly-Man-> now to get those cron scripts to know
<Fly-Man-> and understand when they should be run
<Fly-Man-> then we can make another page on the Help how to set those up
<maxb> wgrant and I are running exactly the same code on Karmic with Python 2.4, and the tests pass for him but fail for me
 * Fly-Man- grins
<Fly-Man-> Hmm, for some reason I just started the code importer .... *laughs*
<jmux> james_w: But where do I get the content of ~/syncs from?
<james_w> jmux: by running update-sources
<james_w> ~/syncs contains the output of the script, not the script itself
 * Fly-Man- laughs
<Fly-Man-> kfogel: Seems that I have the code importer working as well ...
<jmux> james_w: ah ok - so do you have this update-sources script? It's not in the ubuntu-archive-tools
<james_w> I don't think it's part of Launchpad
<james_w> I think it's just a convenience script hacked on top
<james_w> do you have sync-source.py?
<beuno> salgado, the comment for breadcrumbs in your stand up made me both laugh and cry
<jmux> Yes it's in scripts/ftpmaster-tools
<salgado> beuno, but don't worry, I have a plan :)
<beuno> salgado, ...and that's why I like you so much.
<james_w> jmux: update-sources just needs to provide Sources files of the appropriate names for the series that you would like to sync from
<james_w> see the read_Sources function
<jmux> james_w: I know, but I thought this update-sources script would do this downloads for me
<james_w> it would
<james_w> it's just a bunch of wgets though
<Fly-Man-> kfogel: It seems that the local version of Launchpad has the same issues with the SVN import as the website
<kfogel> Fly-Man-: at least now you're in a position to debug :-).
<Fly-Man-> kfogel: Yeah, that's a good thing
<Fly-Man-> because I told wgrant before that with the bzr svn-import it worked like a charm
<Fly-Man-> but the system is using something different
<Fly-Man-> so that's the bottle neck
<Fly-Man-> Anyone, i have succesfully imported a branch on a local Launchpad
<Fly-Man-> but it doesn't show up on the page itself
<Fly-Man-> is there a cron task to run to make it parse ?
<salgado> Fly-Man-, I think that's what the branch scanner does, but I don't know where exactly that script lives
<Fly-Man-> okay, that one I found :)
<EdwinGrubbs2> sinzui: do all pages load the style-3.0.css, or is it just the pages using 3.0 templates?
<sinzui> just 3.0
<sinzui> EdwinGrubbs: 3.0 loaded YUI-reset that destroys most of the style that 2.0 is predicated on
<Fly-Man-> salgado: that branch scanner did notthing
<Fly-Man-> hmm, nope, there seems to be no branch-import being done :|
<rockstar> Fly-Man-, hi.
<Fly-Man-> Hello rockstar
<rockstar> Fly-Man-, push the branch to launchpad, then run the following commands:
<rockstar> bin/py cronscripts/supermirror-pull.py
<rockstar> make scan_branches
<Fly-Man-> rockstar: that looks like the thing I needed ;)
<rockstar> Fly-Man-, I thought so.  :)
<Fly-Man-> rockstar: does that need to be run by cron as well ?
<rockstar> Fly-Man-, it is on production, but for development, I'd only manually run it.  You shouldn't have to push branches that often.
<rockstar> If you're trying to write a test, there are bettor tools for writing the test.
<Fly-Man-> rockstar: It's not a test ;)
<Fly-Man-> Trying to update the Wiki for the ppl that want to use the Launchpad to pull their stuff in
<rockstar> Fly-Man-, ah, as in, a separate instance?
<Fly-Man-> and so far, i'm collecting about 150 OOPS messages :p
<Fly-Man-> rockstar: Have you read the Running page ?
<Fly-Man-> https://dev.launchpad.net/Running
<Fly-Man-> at the end of the page
<Fly-Man-> we made a new page
<Fly-Man-> https://dev.launchpad.net/Running/RemoteAccess
<Fly-Man-> how you can get the LaunchPad to show up on the outside of the system
<Fly-Man-> so you can play with it from another pc
<rockstar> Fly-Man-, see https://dev.launchpad.net/Code/HowToUseCodehostingLocally
<rockstar> Fly-Man-, I'm sure I've seen how to run it to show up on the outside of the system. It's just some apache config.
<Fly-Man-> And the update on the client side through hosts file or DSN entries
<Fly-Man-> But I see some nice things that I didn't know on that page
<Fly-Man-> but good that you are here rockstar
<Fly-Man-> because I think you have the solution to another problem i'm having
<Fly-Man-> with the GIT and SVN pulls on the launchpad.net
<rockstar> Fly-Man-, I'm not here for too much longer today.
<Fly-Man-> rockstar: K, then I will keep it short
<Fly-Man-> There's a bug report in from me
<rockstar> Fly-Man-, what's the bug number?
<Fly-Man-> grabbing
<Fly-Man-> https://bugs.edge.launchpad.net/launchpad/+bug/412974
<mup> Bug #412974: git import failing on project opensim <Launchpad itself:New> <https://launchpad.net/bugs/412974>
<Fly-Man-> thanks you mup :)
<rockstar> Fly-Man-, I'll take a look at it.
<Fly-Man-> rockstar: thank you very much :)
<Fly-Man-> Your page also fixed some of the problems I had with bazaar :)
<Fly-Man-> so thanks again rockstar for that page
<dobey> can someone look at OOPS-1322EC233 ?
<dobey> i am having one hell of a time trying to view the branches i own :(
<matsubara> dobey, bug 396593
<mup> Bug #396593: Person branch listing page timing out <timeout> <Launchpad Bazaar Integration:Fix Committed by thumper> <https://launchpad.net/bugs/396593>
<matsubara> dobey, should be rolled out tomorrow to edge
<dobey> matsubara: ah ok, cool
<dobey> thanks
<matsubara> np
<mrevell> see you Monday guys
<Fly-Man-> mrevell: Have a good weekend :)
<mrevell> thanks, you too :)
<james_w> anyone know what "python setup.py test" does different to just running nosetests?
<james_w> I'm getting failures with the former that I don't get with the latter
<james_w> with wadllib, sorry
<james_w> ah, it seems to be going via unittest
 * kfogel is away: looooooooonch
<Fly-Man-> have a good lunch :)
<Fly-Man-> This doesn't sound good:
<Fly-Man-> ./rocketfuel-setup: line 372: 19483 Segmentation fault      bzr branch http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/ $LP_TRUNK_NAME ERROR: Unable to create local copy of Rocketfuel trunk
<beuno-lunch> sinzui, could you give deryck and myself some insights on bug 413611
<mup> Bug #413611: Convert the comment add templates to 3.0 UI <Launchpad Bugs:In Progress by deryck> <https://launchpad.net/bugs/413611>
<Lantash> @danilos: Would you mind telling me what triggers the import of a translation file after its approval? I'd like to test-drive my fix for #392154.
<mup> Bug #392154: translator-credits show two contributor lists <Launchpad Translations:Triaged> <https://launchpad.net/bugs/392154>
<danilos> Lantash: cronscripts/rosetta-poimport.py will take care of it, but note that file should have been uploaded as 'Published' upload
<danilos> Lantash: 'User' uploads are basically like editing through the web UI
<james_w> garr, that's annoying
<james_w> the problems are cause by discrepancies between simplejson's python and C implementations of the same code
<james_w> http://code.google.com/p/simplejson/issues/detail?id=40
<salgado> beuno, is it intentional that some links to people on https://code.edge.launchpad.net/~salgado/launchpad/cached-menus/+merge/9599 point to lp.net/~person and others point to code.lp.net/~person?
<beuno> salgado, no
<beuno> bug
<beuno> sinzui is suppose to make that all better, I think
<salgado> beuno, where should they all point to?
<beuno> salgado, lp.net/~person
<sinzui> beuno: salgado The fix is landed.
<salgado> sinzui, what's the fix?
<sinzui> beuno: salgado The fmt:link DTRT. fmt:link:bugs willl force a link to pugs for person and pillars. fmt:url is unchanged because when you decide to mess with the URL or text, you have committed to thinking about the issues
<sinzui> salgado: I have not had time to announce it.
<sinzui> broken links are broken because the TAL is not using the official way we make links to people
<sinzui> salgado: you can also use fmt:url:bugs to make a url to the bugs app
<salgado> sinzui, that's a nice change, but fmt:url seems to behave the same way.  at least for people
<salgado> or am I reading it wrong?
<sinzui> salgado: fmt:link defaults to mainsite. fmt:url does not. My reasoning is that since we are only supposed to use fmt:url in rare cases, we probably do not want default ebhaviour
<sinzui> salgado: fmt:url:mainsite will do what fmt:link does for person
<salgado> that makes sense, but the signature of PersonFormatterAPI.url is "def url(self, view_name=None, rootsite='mainsite'):"
<salgado> sinzui, ^
<sinzui> huh!
<sinzui> salgado: I think I screwed up.
<sinzui> salgado: I did intend to make URL do that and for pillars too. I decided after looking at several examples that when we work with fmt:url, we have already rejected the default behaviour. I thought I reverted all def url()
<salgado> sinzui, indeed, it's only for people/teams that url() behaves this way
<sinzui> No test broke. That is interesting in itself
<sinzui> Well I'll prepare a fix if you do not.
<salgado> sinzui, I won't have time for that today
<Lantash> danilos: Thanks alot for your help!
<Lantash> Is it possible to run bin/lint.sh on a 9.04 machine somehow? And running the full test suite on the Amazon EC2 cloud doesn't seem to be possible for external contributors either (it's highly unprobably that the fix breaks any tests though). Would you recommend me to propose a merge for ~lantash/launchpad/bug-392154? Am I supposed to send the Contributor Agreement PDF to you or to Kiko Reis?
<sinzui> salgado: that is fine I can get to it by Monday
<Ursinha> I love the song of commit 9114
<salgado> beuno, do you have some time to talk about breadcrumbs?
<beuno> salgado, I do. IRC or skype?
<salgado> beuno, skype. should be quick
<beuno> salgado, sure, give me 15'?
<salgado> sure
<sinzui> barry: I updated your /people bug
<beuno> salgado, https://edge.launchpad.net/ubuntu/jaunty/+lang/af
<salgado> launchpad.net/tangocms/trunk/+pots/tangocms-aliases/es/+translate
<EdwinGrubbs> sinzui: Why does the base-layout.pt include the context/title in the heading slot, when the location portlet already has the context/title from the fmt:location_heading?
<sinzui> EdwinGrubbs: they will not be the same when printed
<EdwinGrubbs> sinzui: huh, they are the same on the edit pages.
<sinzui> EdwinGrubbs: beuno We need to change s names in the base-layout. because the parts are not what we intended
<sinzui> EdwinGrubbs: beuno: The location portlet is a branding-watermark. The heading-slot is now the final breakcrumb location
<sinzui> EdwinGrubbs: See: https://launchpad.dev/firefox/trunk/+edit
<sinzui> firefox branding, trunk series location, page_title or task it Edit
<beuno> uhm
<beuno> what?
<sinzui> beuno: I think every engineer mis uses the heading-slot because they think that is where the page_title goes to make the <h1> and <title>
<sinzui> beuno: I have this hole in my distro page. I really want to add one more portlet that is guaranteed to be present for all distros. Is there something  from the page you want to see.
<beuno> sinzui, ah
<beuno> how can we fix that?
<barry> sinzui: thanks i saw that.  probably will end up finishing that monday morning
<sinzui> beuno: cprov: I moved the package search into a nice portlet. I see there is a very nice ppa search form. IS there user demand to search ppas from the ubuntu page?
<beuno> sinzui, there's nowhere else to do so
<sinzui> may be we need  a section that points to ubuntu.com to get ubuntu and flavors
<sinzui> oh!. We do not say anything about kubuntu on the ubuntu page.
<cprov> sinzui: yes, flavour do not exist in soyuz (UI)
<gary_poster> salgado: hey.  have you thought about the policy for how to manage source for our non-standard distributions (the one-offs we produce, like we did briefly for Storm and that I've done for other reasons) that kiko-afk asked us to talk about?
<gary_poster> do you want to talk about that sometime soonish?  this evening, or Monday or something?
<salgado> gary_poster, I haven't, sorry.  Monday sounds good to me
<gary_poster> salgado: np.  ok cool.  ping me sometime then, or I'll ping you.
<beuno> barry, https://staging.launchpad.net/bzr
<beuno> barry, try to edit the programming language
<barry> beuno: somebody br0x0red my beautiful ui
<beuno> barry, yes, just a heads up as it will hit edge tomorrow  :)
<barry> beuno: good thing tomorrow is saturday!
<beuno> barry, no it's not
 * barry might as well file the bug for it now though
<beuno> it's friday+1
<barry> beuno: monday-2
<beuno> see, I didn't want it to sounds depressing
 * barry invokes warsaw's second law
<beuno> damn
<barry> beuno: in all seriousness, i'm putting money on it being a bug in lazr-js
<beuno> barry, probably
<beuno> some things still seem to be too fragile
<barry> s/some things/all of javascript/
<beuno> heh
<barry> beuno: on the bright side, the page otherwise looks great :)
<barry> see, i can be optimistic
<barry> lpbug 413793
<mup> Bug #413793: inline editing doesn't play nicely with launchpad 3.0 UI <LAZR Javascript Library:New> <https://launchpad.net/bugs/413793>
<beuno> barry, yeah, I'm lovin 3.0
<beuno> rockstar, thumper, I've spent my afternoon working on the branch index page
<beuno> maybe I'll get it into a RFC state
<beuno> by the end of the day
<beuno> if not, early next week
<flacoste> mthaddon: can you set the default review team of https://code.edge.launchpad.net/~launchpad-pqm/canonical-identity-provider/trunk to the ~launchpad?
<wgrant> maxb: I just ran the entire test suite, after giving it enough RAM and disk. Everything passed.
<wgrant> I have both ~launchpad/ppa and ~maxb/launchpad, with no other custom bits in the stack. Is that your config?
#launchpad-dev 2009-08-15
<maxb> wgrant: I don't have ~launchpad/ppa but given it has nothing published for karmic, that shouldn't matter
<maxb> meh
<maxb> oh well, I can either treat this as cause to dig into the code, or I can treat those as known failures regradless and start trying to get some of my extant changes landed
<wgrant> maxb: I have the ~launchpad/ppa's jaunty enabled.
<wgrant> maxb: Let's see what I have installed from there...
<maxb> hmm
<maxb> lxml and setuptools updates, it looks like
<wgrant> maxb: Something like that. It's hard to tell what's from which PPA.
<wgrant> After activating those PPAs, I upgraded, installed python2.4 and python-celementtree, and built LP.
<wgrant> And it all magically worked.
<poolie> hello wgrant
<wgrant> Hi poolie.
<wgrant> poolie: That's a strange hostname for you...
<poolie> mm
<poolie> i'm at coscup.org
<poolie> in Taipei
<wgrant> poolie: Ahh.
<wgrant> Nice new footer.
<wgrant> Is launchpad-buildd still part of RF? The version there is rather old. I'm wondering because it doesn't work properly with karmic as a host, and it'd be best to fix a non-obsolete version...
<cprov-afk> wgrant: yes, it is (lib/canonical/buildd).  It probably doesn't work for karmic as it is, I have a patch from Adam.
<wgrant> cprov-afk: Right, I know it's there, but from what I've seen there have been a few changes in production since the version in LP.
<cprov-afk> wgrant: yes, 3 new versions IIRC. I'm pushing a branch, so you can check.
<cprov-afk> wgrant: lp:~cprov/launchpad/lp-buildd-karmic
<wgrant> cprov-afk: Ah, excellent. Thanks.
<wgrant> Everything else is now working perfectly on karmic.
<cprov-afk> wgrant: re. bug #413922, it does render correctly (somehow), right ?
<mup> Bug #413922: "supported version of Ubuntu" link on ArchiveActivateView broken <Soyuz:In Progress by cprov> <https://launchpad.net/bugs/413922>
<cprov-afk> wgrant: it's just a misleading TAL instruction.
<wgrant> cprov-afk: It renders a link to something like '/~person/<a href="https://launchpad.net/ubuntu>Ubuntu</a>'.
<wgrant> So no, it doesn't work.
<wgrant> edge is really out of date -- it was broken in the 3.0 migration.
<cprov-afk> wgrant: oh, f.. yes, edge is old.
<wgrant> Really old.
<cprov-afk> wgrant: ha, staging is up.
<wgrant> Barely..
<wgrant> The problem is indeed visible there.
<cprov-afk> wgrant: right ... works for this specific prob.
<wgrant> Along with a conflict between the 2.0 and 3.0 styles, which should be fixed once person-ppas is 3.0ised.
 * cprov hides somewhere.
<wgrant> I suspect this new lp-buildd will fail in a similarly obscure way, but let's see...
<wgrant> cprov: Ah, interesting. That fixed the particularly confusing problem, although it still doesn't quite work without a patch.
<wgrant> (dpkg-source's output changed recently. Debian's sbuild was patched to no longer parse its output 18 months ago, but this sbuild seems muuuch older than that.)
<maxb> Why does the dev setup use two different IP addresses? It appears to just be in support of running bazaar.launchpad.dev on a separate IP - why is that?
<wgrant> maxb: the bazaar vhost is completely different from the others.
<wgrant> But it still needs SSL.
<wgrant> And SSL doesn't do multiple vhosts on one IP address.
<wgrant> (unless you use SNI, which Ubuntu doesn't support yet)
<maxb> Why does the bazaar vhost need SSL?
<maxb> http://bazaar.launchpad.net/ seems perfectly happy to talk to me
<wgrant> Because it does in production, to protect cookies.
<maxb> Oh, loggerheading of private branches?
<wgrant> (for private codebrowse)
<wgrant> Right.
 * maxb is attempting to make
<maxb> oops
 * maxb is attempting to make dev.lp.net/Running/RemoteAccess saner, and would be happy to drop the requirement for multiple IPs
<wgrant> Maybe in a release or two.
<maxb> Well, dev.lp.net/Running/RemoteAccess is a bit of a hack anyway :-)
<wgrant> Indeed.
<wgrant> maxb: Still no luck with getting tests to pass?
<maxb> Nope, I think it's dig-into-the-code time
<wgrant> Damn.
<maxb> I just need to decide whether I do that now, or just ignore those two tests and press on with landing some of the py2.5 changes I've already accumulated
<maxb> Switching topics: Is there a statement from Canonical explaining its rationale in choice of license for Launchpad icing/images?
<wgrant> kfogel might have mentioned it in a comment on a blog post, but I don't recall anything official.
<wgrant> Although it's pretty obvious why.
<maxb> Well, there's multiple aspects and it's not entirely obvious.
<wgrant> That's true. I can think of two big ones.
<maxb> For example, I'd love to use Malone for both non-profit private projects, and attempt to convince my company to use it.
<maxb> Neither of those are ever going to be revenue stream for Canonical, nor damage Launchpad's cohesive one-place-for-OSS position
<wgrant> The second could be a revenue stream, if companies get on the ooh-cloudy bandwagon.
<maxb> If I can't do those, it would be nice to know why exactly :-)
<maxb> My company will never buy into putting its bugtracking info on a 3rd party server
#launchpad-dev 2009-08-16
 * maxb pretty much rewrites dev.lp.net/Running/RemoteAccess entirely
<wgrant> maxb: A good idea.
<maxb> It was a bit horrid
<wgrant> maxb: Ah, much better.
<thumper> morning
<maxb> evening! :-)
<thumper> spm: ping
<spm> thumper: heyo
<thumper> spm: morning
<thumper> spm: what's up with staging?
<spm> that'll learn me to not check the 'red' tabs sooner - assumed it was a weekend ping and could wait a bit... :-)
<spm> thumper: looks dead. poss, staging restore boomsville
<thumper> :)
<spm> thumper: yah. looks like the shutdown/restore part went a tad haywire. looks good now.
<thumper> spm: ok, when is the next code update?
<thumper> spm: I landed some stuff in the weekend
<thumper> that I wanted to check
<thumper> but I need r8385 :)
<thumper> just one more than there is there now
<spm> thumper: the latest was about 20ish hours ago; but part of is still running.
<lifeless> timeouts on lp-code?
<lifeless> https://code.edge.launchpad.net/bzr
<thumper> lifeless: there is a fix coming
<thumper> lifeless: it has landed
<thumper> I could request a CP I guess
<thumper> lifeless: some of these are due to the rapid growth of new (package) branches
<lifeless> scaling is hard ;)
<thumper> it is
<lifeless> eat more
<lifeless> scale sideways
<lifeless> it works
<thumper> heh
<thumper> we are
<thumper> with the db
<wgrant> How many DB servers with 128GiB of RAM are there now? Three?
<thumper> not sure
<thumper> but they are chunky servers
<wgrant> Yet LP remains slothful :(
<thumper> wgrant: ORMs suck
<thumper> wgrant: I know that is where much of our problems come from
<thumper> I've been on a query optimisation drive
<wgrant> thumper: Indeed.
<wgrant> It would be nice to have a large sample data set to see where things go wrong. But I guess you guys have (soft) timeouts for that.
<lifeless> wgrant: file a bug
<lifeless> seriously, there is no reason we can't look at some programmatic transformation of real data to make a <large> test set, such things have been done internally before
<thumper> lifeless, wgrant: that has been on someones TODO list since I started almost three years ago
<beuno> hi thumper
<lifeless> thumper: now we have volunteers!
<thumper> hi beuno
<thumper> beuno: I have comments about the new design
<beuno> thumper, fantastic
<thumper> beuno: and questions
<beuno> thumper, great, I wanted to kick off a discussion mostly
<thumper> beuno: yeah
<thumper> beuno: I've been doing a lot of conversion stuff
<thumper> beuno: mostly the listing and editing pages right now
<thumper> beuno: I have a branch to change active reviews
<thumper> beuno: to include the approved merges
<beuno> ah, great
<thumper> beuno: I also want to talk to you about the review stuff
<thumper> beuno: and ideas I've got
<thumper> beuno: but I seriously feel I don't have enough time to spread myself around
<beuno> thumper, yeah, everyone in your team thought "UI cycle" meant "vacation cycle"   :)
<thumper> beuno: yeah
<thumper> beuno: I've been doing the code ones, and Paul has been doing answers
<beuno> thumper, any ideas what's holding up the landing of answers?
<beuno> I see you've been doing quite well with code
<thumper> beuno: no, I'll be checking in with rockstar tomorrow morning my time, and we'll get some traction
<beuno> great
<thumper> beuno: I know he has done a lot of work on it, and just needs some nudging to get bits landed
#launchpad-dev 2010-08-16
<jelmer> mwhudson: unfortunately they don't get propagated
<mwhudson> jelmer: oh really?
<mwhudson> how completely useless
<jelmer> mwhudson: I discussed them with some git developers at LCA, and we came to the conclusion that git notes wouldn't be useful for this sort of thing.
<mwhudson> oh ok, i got the idea somehow that they were a new thing
<jelmer> They are relatively new, only about a year I think.
<jelmer> I ended up implementing bzr-git roundtripping using extra metadata in commit messages (revision properties, revision id) and a file-id table in the tree.
<mwhudson> ah ok
<lifeless> jelmer: so
<lifeless> /home/robertc/launchpad/lp-branches/working/lib/lp/scripts/utilities/importfascist.py:187: DeprecationWarning: please use 'debian' instead of 'debian_bundle'
<lifeless>   module = original_import(name, globals, locals, fromlist, level)
<lifeless> jelmer: when is that getting stabbed ?
<mwhudson> jelmer: i see the kdebase import failed :/
<poolie> lifeless: yes that is nice to separate the policy bits of testresources from the mechanism
<jelmer> lifeless: I thought I already head
<jelmer> *had
<lifeless> jelmer: I
<lifeless> 'm on latest devel
<lifeless> latest sourcedeps, upgraded my packages
<lifeless> hmm, I'll hit apt harder there were some hold- backs
<jelmer> lifeless: it doesn't tell you where that DeprecationWarning is coming from?
<jelmer> mwhudson: yeah :-(
<lifeless> jelmer: no, its a little opaque
<mwhudson> jelmer: any idea on this one?
<jelmer> lifeless: Have you run update-sourcecode recently?
<lifeless> jelmer: never ?
<mwhudson> seems that it's some kind of remote server problem
<jelmer> mwhudson: yeah, it's the same one as the len(tview) != len_tview one
<mwhudson> jelmer: oh ok
<lifeless> poolie: small request for you
<lifeless> poolie: I'd like to put a rule in the feature flags ruleset on production
<poolie> ok
<lifeless> which will say 'beta users team membership means is_edge should evaluate true'
<poolie> by all means
<poolie> so keeping the same behaviour, but changing the guts to use the flags mechanism rather than being hardcoded?
<lifeless> yeah
<poolie> yes i was thinking of doing that soon too
<lifeless> for now is_edge (and is_lpnet) need to union the two things
<lifeless> as a transition, I think.
<lifeless> e.g.
<lifeless> if the appserver is configured as edge
<lifeless> then is_edge is true and is_lpnet is false
<lifeless> if the appserver is configured as lpnet
<poolie> so last thursday
<lifeless> then the flags rule can override that
<poolie> which seems like a long time ago
<poolie> i got a readonly web view of them
<lifeless> to say 'actually, this is edge, nyarh'
<poolie> and i was trying to work out a nice way to make them editable
<lifeless> poolie: I quite like what sinzui has organised for jabber ids with bac
<lifeless> which is, you can add or delete, but not edit in place; it makes it kindof simple
<lifeless> I dunno - we can have something pretty crude - even a big textfield you parse and diff would do :). I don't know this part of the LP machinery as yet.
<poolie> yeah, something like that
<jelmer> mwhudson: getting that particular bug fixed is high on my todo list, but it's non-trivial
<poolie> it wasn't a blocker, that was just what i got up to before i stopped
<mwhudson> jelmer: it's fixable on the bzr-svn end?
<jelmer> mwhudson: yeah, it's a bzr-svn bug
<mwhudson> oh ok
<jelmer> mwhudson: the problem is that because of odd operations in the repo bzr-svn ends up with an invalid base text to apply the delta it receives from the server against
<mwhudson> jelmer: is it like the hash reconstruction fun you had with bzr-git?
<jelmer> mwhudson: It's just for the file fulltext, doesn't depend on the particular serialization that bzr-svn uses
<mwhudson> oh ok
<mwhudson> sounds like fun :)
<jelmer> s/bzr-svn/svn/
<lifeless> jelmer: still up ?
<lifeless> or StevenK  ?
<lifeless> I have this happening :
<lifeless> robertc   6840  0.0  0.2  26224  4276 pts/1    T    10:52   0:00          \_ /usr/bin/perl -w /usr/bin/debuild --no-conf -S -k0x5D147547
<lifeless> robertc   6868  0.0  0.0   5572   772 pts/1    T    10:52   0:00              \_ tee ../biscuit_1.0-4_source.build
<lifeless> robertc   6929  0.0  0.0  19404  1920 pts/1    T    10:52   0:00              \_ /bin/bash /usr/bin/debsign -k0x5D147547 biscuit_1.0-4_source.changes
<lifeless> robertc   6968  0.0  0.0   5596   772 pts/1    T    10:52   0:00                  \_ stty 400:1:bf:a20:3:1c:7f:15:4:0:1:0:11:13:1a:0:12:f:17:16:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0
<lifeless> ^ hung test
<lifeless> jelmer: the stty is looping:
<lifeless> ioctl(0, SNDCTL_TMR_STOP or TCSETSW, {B38400 opost isig icanon echo ...}) = ? ERESTARTSYS (To be restarted)
<lifeless> --- SIGTTOU (Stopped (tty output)) @ 0 (0) ---
<lifeless> -forever-
<lifeless> --- SIGTTOU (Stopped (tty output)) @ 0 (0) ---
<lifeless> man, my vim in this vm is _borked_
<lifeless> its the cause, whatever is going on.
<lifeless> mwhudson: did you have any thoughts on https://bugs.edge.launchpad.net/launchpad-foundations/+bug/618019 ?
<_mup_> Bug #618019: OOPS may be underrepresenting storm/sql time <Launchpad Foundations:New> <https://launchpad.net/bugs/618019>
<mwhudson> lifeless: not particularly
<mwhudson> lifeless: i know that object construction time can be significant
<mwhudson> calling it 'sql time' doesn't seem quite fair though
<lifeless> well, I'm saying we don't know
<lifeless> I'd like to be able to say ORM time and sql time
<lifeless> but I have a sinking feeling tha time spent getting stuff out of the sql socket is being accrued as nonsql time
<lifeless> mwhudson: ould factory.makePerson be reusing Person db id's ?
<mwhudson> lifeless: no
<lifeless> I have a -weird- interaction then
<mwhudson> there might be db-non marked dirty issues
<mwhudson> lifeless: details++ pls ;)
<lifeless> yes
<lifeless> typing
<lifeless> lp/registry/browser/tests/coc-views.txt
<lifeless> line 37
<lifeless> makes a new person
<lifeless> grabs a view
<lifeless> and checks that the instructions are show
<lifeless> n
<lifeless> I'm seeing
<lifeless>     - 1. Register an OpenPGP key.
<lifeless>     - 2. Download the current Code of Conduct.
<lifeless>     - 3. Sign it!
<lifeless> which means the view things the principle has signed the coc
<lifeless> I think
<wgrant> lifeless: Wouldn't that suggest the opposite?
<lifeless> wgrant: the - means the line is missing
<wgrant> Oh.
<wgrant> Not a bullet.
<wgrant> I see.
<lifeless> now, for hilarity
<lifeless> I can't find those instructions in the source
<wgrant> What are the changes in your branch?
<lifeless> wgrant: its the registry branch
<wgrant> The one where you're caching that field?
<lifeless> yes
<mwhudson> lifeless: the instructions are in codeofconduct-list.pt
<lifeless> nothing obvious in a preview merge
<lifeless> oh, the </a> threw my gre out
<lifeless> mwhudson: thanks
<lifeless> it also seems to think that they have registered an openpgpg key
<lifeless> which is -really weird- as I didn't cache that
<lifeless> ah no, its all in the not: is_ubuntu_coc_signer clause
<lifeless> so the symptoms are 'a new person from factory.makePerson has their is_ubuntu_coc_signer set true
<lifeless> yes, I added a print of the is_ubuntu_coc_signer right after makePerson
<lifeless> -> True
<wgrant> lifeless: Is it caching (eg. does it work in make harness), or is your query broken?
<lifeless> my query ?
<lifeless> wgrant: this isn't going through _all_members
<lifeless> wgrant: thats a -very- explicit code path;
<wgrant> lifeless: But you refactored the property itself.
<lifeless> yes
<lifeless> wgrant: oh, right, I see what you're asking. Thanks.
<wgrant> I haven't seen this AND thing before.
<lifeless> thats easy to tes
<wgrant> Ah.
<wgrant> There's the problem.
<wgrant> You're not constraining the Person...
<wgrant> You're just saying Person.id.
<lifeless> yeah, its the calculation itself
<wgrant> You mean self.id.
<wgrant> Ah, except that it's static.
<lifeless> yeah, the refactoring is borked
<lifeless> very good catch
<lifeless> in both cases in fact.
<lifeless> not enough attention to detail; fixing.
<wgrant> Hah.
<lifeless> it needs to LeftJoin on the _all_members case
<lifeless> to be fair though, it was the second attribute, I was still figuring things out ;)
<wgrant> Hm, is the _all_members case really broken?
<lifeless> yes
<wgrant> I'm not sure how Storm will SQLify that.
<wgrant> OK.
<lifeless> people that haven't signed a coc at all will be excluded
<lifeless> because their columns would be NULL
<wgrant> But it's in an Exists...
<lifeless> oh
<lifeless> I think I need more caffeine. This isn't me :)
<lifeless> right. square one.
<wgrant> Heh.
<lifeless> the _all_members case does look right.
<wgrant> I think it's OK, yes.
<lifeless> the property is naffed
<lifeless> we can't use Person.id there
<lifeless> so
<wgrant> You can.
<wgrant> You just have to constrain it in the property itself.
<lifeless> we're not querying on the Person table
<lifeless> and querying both tables would be nuts
<wgrant> True.
<wgrant> brb.
<lifeless> mwhudson: wgrant: thank you for your help.
<lifeless> taking a break; -> new house stuff and talking to bigjools late tonight
<lifeless> if you need me, SMS/ring the aussie mobile
<lifeless> getting there with this branch
<wgrant> The cache-the-world one?
<lifeless> cache person
<lifeless> got some wtf failures in soyuz tests
<lifeless> also in my cachedproperty branch which I'm using as a prereq now
<lifeless> wgrant: for instance
<lifeless> File "lib/lp/soyuz/browser/tests/archive-views.txt", line 1279, in archive-views.txt
<lifeless> Failed example:
<lifeless>     view = create_initialized_view(
<lifeless>         ubuntu_team.archive, name="+copy-packages",
<lifeless>         form={
<lifeless>             'field.destination_archive': '',
<lifeless>             'field.destination_series': '',
<lifeless>             })
<lifeless> ...
<lifeless>         raise ComponentLookupError(objects, interface, name)
<lifeless>     ComponentLookupError: ((None, <canonical.launchpad.webapp.servers.LaunchpadTestRequest instance URL=http://127.0.0.1>), <InterfaceClass zope.interface.Interface>, '+copy-packages')
<lifeless> has me scratching
<StevenK> lifeless: I'm also seeing a hanging test, FWIW
<lifeless> StevenK: in soyuz? same symptoms? how are you running it ?
<StevenK> lifeless: I threw a branch at ec2 land
<StevenK> ec2test@domU-12-31-39-0E-60-31:~$ tail -n 1 /var/www/current_test.log && date
<StevenK> time: 2010-08-16 05:14:30.759898Z
<StevenK> Mon Aug 16 07:07:44 UTC 2010
<StevenK> lifeless: strace is being as unhelpful as I feared
<lifeless> \o/
<lifeless> so
<lifeless> what does ps tell you ?
<lifeless> e.g. ps fux
<StevenK> All that shows is the shell, the ps process and the librarian
<lifeless> hmm, what user are you :)
<lifeless> oh, no terminal
<StevenK> ec2test
<lifeless> ps faux ?
<lifeless> the stty process is what was hurting me
<lifeless> (and its a known 'feature' of stty
<StevenK> No stty process
<lifeless> what do you have?
<StevenK> lifeless: http://pastebin.ubuntu.com/478711/
<StevenK> I'm suspecting the test runner has fallen over
<lifeless> well
<lifeless> did you see the patch from maris
<lifeless> shutdown stomps on shutdown
<lifeless> so falling over is possible
<lifeless> finishing and not shutting down is also possible
<StevenK> I seriously doubt the test suite finished
<lifeless> subunit-ls < test.log
<lifeless> sorry
<StevenK> Minusing the last test ran versus what time the instance started is 80 minutes.
<StevenK> We're not that quick :-)
<lifeless> subunit-stats < test.log
<lifeless> true
<StevenK> It only ran 1744 tests
<StevenK> One did fail, though, which is odd
<lifeless> may be interrupted
<lifeless> if the test kills the runner thats how it shows up
<StevenK> error: lp.soyuz.tests.test_buildpackagejob.TestBuildPackageJob.test_providesInterfaces [
<StevenK> _StringException: lost connection during test 'lp.soyuz.tests.test_buildpackagejob.TestBuildPackageJob.test_providesInterfaces'
<StevenK> lifeless: Like so, from subunit-filter ?
<lifeless> yes
<lifeless> that tells you what nuked things
<lifeless> assuming no buffering
<StevenK> Indeed
<lifeless> [which is false - I *know* we have buggering in place in the test supervisor]
 * StevenK smirks
<lifeless> s/gg/ff/
<StevenK> lifeless: So, what do you suggest? Kill the test runner and run make check by hand on the instance?
<lifeless> well
<lifeless> is it repeatable?
<StevenK> This branch has done this twice, but I don't know which test killed it the first time since I was sleeping
<lifeless> running the tests while watching isn't a silly idea
<lifeless> you could try running just that test
<StevenK> Running just that test locally doesn't fail
<StevenK> But, like you say, I think buffering is screwing us
<lifeless> you can run it and the next 20
<lifeless> grab any previous ec2 result
<lifeless> and use subunit-ls to get a list of the tests
<lifeless> pick that one and 20 or 30 after
<lifeless> put them in a file and use bin/test --load-list filename
<StevenK> The ordering doesn't change?
<lifeless> its tolerably stable
<lifeless> btw
<lifeless> 1500 line text files are not 'tests'
<lifeless> just saying
<StevenK> Wha?
 * StevenK is missing contextg
<StevenK> s/g$//
<lifeless> tests/archive-views.txt
<lifeless> its blowing up, spectacularly, for me
<lifeless>       File "/home/robertc/launchpad/lp-sourcedeps/eggs/zope.component-3.9.3-py2.6.egg/zope/component/_api.py", line 111, in getMultiAdapter
<lifeless>         raise ComponentLookupError(objects, interface, name)
<lifeless>     ComponentLookupError: ((None, <canonical.launchpad.webapp.servers.LaunchpadTestRequest instance URL=http://127.0.0.1>), <InterfaceClass zope.interface.Interface>, '+copy-packages')
<lifeless> line 1279
<StevenK> lifeless: That test predates me by a while
<lifeless> sure
<lifeless> not blaming you :)
<lifeless> has anyone seen one of these sorts of failures before
<lifeless> and can suggest how to debug the thing ?
<StevenK> He can!
<StevenK> Um, I have, but I can't remember what I did
<noodles775> lifeless: normally you see that kind of error when ZCA hasn't been initialised properly... I'm assuming you've not modified the test so you haven't changed the layer it runs with?
 * StevenK grumbles at the letter he just got from Medibank Private
<lifeless> noodles775: no, I haven't changed anything in soyuz
<lifeless> noodles775: this is my registry branch, which adds some caching to Person (and only Person)
<bigjools> hullo
<lifeless> hey bigjools
<noodles775> lifeless: ah, so it's only failing in your branch? I'll take a look at the MP.
<bigjools> hey lifeless, epic fail at getting up early I'm afraid
<lifeless> bigjools: thats ok
<StevenK> bigjools: Still flu-ey?
<StevenK> bigjools: Read as, "Did the weekend help?"
<bigjools> not quite full strength but better, thanks
<lifeless> noodles775: I'm pushing the latest now, but the shape is unchanged
<lifeless> noodles775: https://code.edge.launchpad.net/~lifeless/launchpad/registry/+merge/32067
<noodles775> Ta.
<lifeless> its pushing now (bit slow because Lynne is eating all the wifi slots with a machine migration)
<noodles775> no worries... I'm just running the doc firstbefore merging anyway.
<noodles775> *sigh* and need to run make schema first.
<lifeless> the cachedproperty changes are from a different branch
<lifeless> which is approved and ec2ing itself
 * StevenK stares at bin/test --load-list on his instance
<StevenK> lifeless: Your suggestion was to run bin/test --load-list <filename>. That has set up the layers and then done nothing else
<lifeless> you probably want -vv there too :)
<StevenK> It just spat out 'Killed', so I'm now *very* curious what is going on
<StevenK> lifeless: Probably :-)
<noodles775> lifeless: the cache you've added on IPerson.archive is changing the test: http://pastebin.ubuntu.com/478724/
<noodles775> (sorry, doc :) L.
<lifeless> noodles775: thanks
<noodles775> np.
<lifeless> (waiting for pakcets to look at the pastebin)
<StevenK> ec2test@domU-12-31-39-0E-60-31:~$ dmesg | grep -c 'oom-killer'
<StevenK> 6
<StevenK> lifeless: ^
<lifeless> -epic- packet loss on local wifi :(
<StevenK> The plot thins :-(
<lifeless> hah
<StevenK>  8693 ec2test   20   0 3575m 3.2g  11m R  100 45.1   4:35.38 /usr/bin/python
<StevenK> Fuuuuuuuuuun
<lifeless> thats less that optimal
<StevenK> bin/test is blaming buildd-slavescanner.txt
<lifeless> noodles775: so what does it /mean/ ?  'no archive found when one should be found' ?
<noodles775> lifeless: I'm guessing it means that ubuntu_team.archive was accessed before the doc added an archive for them, so the value of None is cached...
<noodles775> The line of the error you see is where it tries to adapt ubuntu_team.archive (which is None) and fails.
<noodles775> As the paste shows, the archive is found if you kill the cache (and the initialized view returned as expected)
<noodles775> So, convert it to a unit test and the problem goes away :)
<lifeless> noodles775: yeah, I need to find where though - so that I can fix the code to invalidate automatically (there are many more failures, this is just the first bit of fallout that was completly bizarre)
<lifeless> bigjools: so
<StevenK> Right, Python got to 6.7g resident before the oom-killer stepped in
<lifeless> bigjools: I don't konw what bits needed expansion in the incident report
<mrevell> Hello all
<lifeless> bigjools: so I suggest you ask me stuff ;)
<bigjools> lifeless: so an explanation of how you reached your conclusion would be a good start
<lifeless> noodles775: ArchiveSet.new() caches the value
<lifeless> bigjools: which conclusion ?
<noodles775> lifeless: aha.
<lifeless> noodles775: yes, aha indeed.
<bigjools> lifeless: "Soyuz went into a busy loop on the 'bohrium' builder"
<lifeless> noodles775: I suspect its this : if purpose == ArchivePurpose.PPA and owner.archive is not None:
<noodles775> lifeless: yep. Isn't this why we don't normally cache model attributes? (sorry, I've probably not caught up on some email discussion saying why what you're doing is ok).
<lifeless> bigjools: wgrant and cody talked about that.
<lifeless> bigjools: cody looked at the log on devpad and saw multi-times-per-second logged events saying that bohrium was being disabled
<lifeless> bigjools: I thought I linked the example log entry in the report
<bigjools> lifeless: that's just one snippet, it doesn't show repeated attempts at anything
<lifeless> noodles775: see several threads of mine about this on the list :) short story: caching is *a* way to get 'things that look cheap are cheap' more widespread, and bigger picture solutions require r&d
<lifeless> bigjools: it was spitting that out a lot, or so I was told
<bigjools> ok
<lifeless> bigjools: other evidence about a busy loop on bohrium is that attempts to update it from both psql and the lp webapp and airlock all failed
 * wgrant was just working on what Cody said was hoppening in the logs.
<bigjools> lifeless: in the traceback it's calling "requestAbort"
<lifeless> specificaly just update builder set thing=False where name=bohrium got stuck waiting for a lock
<lifeless> that was a sign that *something* was busy updating that row....and updating the row.... and updating the row
<lifeless> we *don't know* if it was one long transaction, or many short ones.
<wgrant> bigjools: That codepath then immediately sets builderok=false, then commits. With no other options.
<lifeless> noodles775: pm
<bigjools> wgrant: nope, it calls slave.abort()
<wgrant> bigjools: True. Which fails, then invokes the exception handler which prints the log message.
<bigjools> which is why we get an XMLRPC fault
<wgrant> Which sets builderok=false, which commits.
<wgrant> Missed the exception handler bit, sorry.
<bigjools> ah ok
<wgrant> But we know that the handler was called, since the log entry is there.
<lifeless> noodles775: does that pasted thing look reasonable to you ?
<lifeless> it makes that entire doctest pass
<wgrant> In fact, I think that's just about all we *know* happened.
<lifeless> we also knew that other builds were not happening
<wgrant> True.
<lifeless> (or were being updated/processed so slowly that it was equivalent to not happening)
<wgrant> Now, I haven't seen the logs, but from what I heard there was nothing about the commit failing.
<wgrant> So possible something kept setting builderok=true again, or the Twisted evil is swallowing exceptions.
<bigjools> lifeless: so what that means is that we never got back into the reactor, so there was a loop somewhere
<bigjools> lifeless: was the log spewing stuff or stuck on that line?
<noodles775> lifeless: hrm... it looks like, yes, it would get the doctest passing, but it's adding a dependence elsewhere on the knowledge that a property is cached... would something like @cachedproperty(unless_equal=None) be silly? Hrm
<lifeless> bigjools: I didn't look : when I got here it had been plausibly analysed > 1 hour before, without escalation: so I discussed enough to determine that it needed (IMO) escalation, and handed off to elmo
<lifeless> noodles775: yes, because it would mean that /participants of a team with many people without PPAs would trigger lots of tiny queries
<noodles775> lifeless: Right.
<noodles775> lifeless: So I can't think of any other solution immediately than what you've pasted.
<lifeless> archive.txt does this
 * StevenK tries to figure out why a doctest is causing python to eat 6.8g of RAM
<lifeless> mm, does something similar
<bigjools> lifeless: ok so I am looking at the log now, it's repeating that log section over and over
<bigjools> lifeless: I have a suspiscion that it happened when the Enablement guys yanked it from the pool
<bigjools> assuming they did so on a Saturday
<bigjools> actually, Friday night
<lifeless> you say potato, I say pohtahto
<bigjools> b-m was stuck in this loop from 22:17 to 08:47
<bigjools> no you don't :)
<lifeless> I mean tz issues :)
<bigjools> that's UTC
<lifeless> yeah
<lifeless> so 10am sat
<bigjools> therefore it was Friday night everywhere ;)
<wgrant> Oho. @write_transaction retries....
<bigjools> forever?
<wgrant> Only three times, apparently.
<bigjools> was gonna say ...
<wgrant> But it's still utterly wrong to use it.
<bigjools> ?
<wgrant> What with the whole talking to the builder thing.
<lifeless> what do you guys think of the idea of making the buildd manager an API client
<bigjools> you need to expand on that
<bigjools> lifeless: CRACK
<lifeless> and removing its DB usage.
<bigjools> total crack
<lifeless> why ?
<wgrant> bigjools: Retrying transactions that have external effects seems... unwise.
<bigjools> ...
<wgrant> Although it's probably OK enough here.
<bigjools> lifeless: the API is S L O W
<lifeless> bigjools: not in any way that matters for the buildd manager
<bigjools> dude
<bigjools> no
<bigjools> it does matter
<lifeless> of coure performance matters
<bigjools> you're just making things twice as hard for yourself and putting load on appservers for no good reason
<lifeless> but nothing the buildd manager *does* has any reason to be slow with the API as it stands today.
<lifeless> no, I'm running an idea up the flagpole
<lifeless> if its a bad idea, fine.
<bigjools> the b-m is *very* busy issuing queries
<lifeless> but I want to understand *why*
<bigjools> first of all, you need to justify why you think it would be better with the API
<lifeless> sure
<lifeless> you've got a highly concurrent task
<lifeless> which twisted is great at
<lifeless> but you're doing transactions, which will - unless you are _very_ careful - last the length of a conceptual task like 'check on builder Y'
<lifeless> the longer transactions are, the more contention you put on busy rows in the DB, like the builder table.
<lifeless> secondly
<bigjools> first one easily fixed by moving transaction boundaries
<bigjools> (it already does partial commits)
<lifeless> our *entire* stack above the storm layer is built on the model of global, magical, transaction objects which lookup information in thread locals
<lifeless> [it may  be easy to fix, but it needs to be done and maintained with care: using the API you'd have that for free, so its less effort *in that regard*]
<lifeless> back to the second angle
<lifeless> twisted runs everything in the reactor thread
<lifeless> so you have to play silly buggers with our stack to move transaction objects out of context and back in, and also, because the stores and transactions are tied to model objets, possibly do that to the model objects too
<lifeless> if you don't, you run a high risk of bugs where you do unrelated things in a single transaction
<lifeless> note that moving transaction boundaries won't work well if you want to use our regular model objects for flow control or anything like that.
<bigjools> lifeless: I don't see why things are moving around as you suggest
<lifeless> I think that those two things together make the job of writing the buildd manager in twistd harder than if it was an API client. Of course, I'm not the one writing it: I'm expressing an opinion and asking what you think.
<bigjools> the b-m would bring the appservers to their knees
<lifeless> why?
<lifeless> I'm not trying to be silly or difficult - I really don't see why it would.
<bigjools> because it issues shitloads of queries
<bigjools> ask stub about the load it generates
<lifeless> what is it doing with these queries ?
<bigjools> seeing if there's a new build on each builder, checking status of each builder etc etc
<bigjools> if the webservice were a thin, performant agile layer I would consider agreeing with you, but it's not
<bigjools> even if it were you'd still be putting tremendous load on the appservers
<bigjools> and I think we should use those to service external requests, not DC ones
<lifeless> so, in terms of its performance; I've been looking and its bad in a couple of specific ways
<lifeless> it batches stuff we shouldn't batch : but clients have control over that, so we can easily avoid it.
<lifeless> and it potato programs as you traverse objects (the exact same terrible pattern that makes some pages (and some API calls) extremely slow)
<lifeless> the former issue would be avoidable; the latter issue is nearly entirely irrelevant in the DC (particularly for a long lived process like the buildd manager)
<lifeless> in terms of appserver load
<lifeless> completely separately from this discussion
<lifeless> I want to separate out APIs and WebUI appservers
<lifeless> I don't think its good or appropriate to mix up human-facing work with machine-driven work: makes servicing humans well in times of overload harder.
<lifeless> this idea is raw: I haven't run it up the flagpole and had it examined it; its an open-concept
<bigjools> ok
<lifeless> anyhow, if that were done, the load on the appservers might be huge, but it wouldn't affect browser using users.
<bigjools> anyway I am trying to see reasons why the b-m would be stuck in a slave.abort() loop
<bigjools> it's not exiting from the Deferred at all, otherwise we'd see other builds getting dispatched in the log
<lifeless> Anyhow, this is a diversion: I asked your opinion, which is that its a bad idea because it would be harder to make it perform well, and even if done to perform well it has a high risk of adversely affecting the API appservers.
<lifeless> I may come back to this concept in the future, but I have what I wanted for now:)
<lifeless> noodles775: actually, hah, it didn't fix it - I've got some more digging to do.
<lifeless> what happens if you removeSecurityProxy on a non proxies object ?
<lifeless> ah, pass though. nice
<lifeless> right, scatter removeSecurityProxy into the caching layer, and its good.
<lifeless> bigjools: is there anything else I can tell you about saturday to help
<lifeless> danilos: hi
<danilos> lifeless, hi
<lifeless> is there anything I can do you help with your page performance analysis ? your plea for help was rather heartfelt :)
<danilos> lifeless, heh, thanks for the offer :) not sure right now, but I just wanted to indicate a few issues that seem as if they are staying under your radar :)
<danilos> lifeless, it's basically: "here's a few things we are having problems with, I know different people are working on it, but I am sure you are the best person to keep that all in mind"
<lifeless> danilos: I've filed a bug - thats related - https://bugs.edge.launchpad.net/launchpad-foundations/+bug/618019
<_mup_> Bug #618019: OOPS may be underrepresenting storm/sql time <Launchpad Foundations:New> <https://launchpad.net/bugs/618019>
<lifeless> \o/ I think my registry branch will pass now
<lifeless> danilos: I think this is related because if we're undercounting DB time, it would certainly explain some things ;)
<wgrant> lifeless: Doesn't mean it won't break anything :/
<danilos> lifeless, right, but this particular case doesn't seem to be that
<lifeless> danilos: do you have a profile for it ?
<danilos> lifeless, for instance, we get long rendering time on local instances with that many objects where DB time is very stable (i.e. we just add 2000 rows to the DB)
<danilos> lifeless, no, sorry, it's the next step we've got to take
<lifeless> no worries
<lifeless> losa ping
<lifeless> :)
<danilos> lifeless, (I've got KCacheGrind email tagged as "important" :)
<lifeless> :(
<lifeless> bah
<lifeless> :) is what I meant
<danilos> heh
<lifeless> danilos: if it is python time, there are a few things we can do to fix it, but which one will make sense will need the data for where the time is going
<danilos> lifeless, also, our first suspicion was storm "objectification", but that turned out to be pretty quick
<danilos> lifeless, of course, we haven't done anything other than get a gut feel about the situation
<lifeless> (we could do a C extension, we can cut some fat out, we can rearrange the layers)
<lifeless> wgrant: of course it will break something.
<danilos> lifeless, gary was mentioning switching to chameleon (as a faster TAL renderer), looking into fmt:url and why it takes so long (it was around 2ms per call for us), etc. anyway, I don't want to make another guess without profiling first :)
<lifeless> \o/ data
<lifeless> bigjools: ok, I'm going to go do family time and stuff
<lifeless> bigjools: if there are other things I can answer to help, or if you want me to start looking at the code as another of eyeballs, just shout.
<bigjools> lifeless: sorry was on a call
<bigjools> lifeless: if you could eyeball the code that would be great.  It's getting  stuck inside a Deferred somehow
<bigjools> I don't know why it would be repeatedly calling slave.abort
<bigjools> well, it's calling builder.rescueIfLost()
<wgrant> The log says it's also repeatedly calling Builder.updateStatus...
<wgrant> And there's only one place that's called :/
<bigjools> wgrant: I only see it calling rescueIfLost
<wgrant> bigjools: The traceback sucks. But grep for the error text.
<bigjools> wgrant: updateStatus is not in the traceback
<bigjools> http://pastebin.ubuntu.com/477761/
<wgrant> bigjools: updateStatus delegates to updateBuilderStatus.
<wgrant> Which is right at the top.
<bigjools> d'oh
<wgrant> NFI why the traceback stops there, though.
<bigjools> right
<bigjools> I still feel it's something to do with Enablement yanking the builder
<wgrant> Very probably.
<wgrant> But even DB contention doesn't explain this entirely.
<wgrant> It explains three calls.
<wgrant> Not thousands.
<wgrant> Unless they're both mutually timing each other out.
<bigjools> lifeless: you said that Builder:+edit is slow, I can't find an oops on bohrium in the oops reports
<lifeless> it was on sat
<lifeless> only three requests, and I think they were all elmo
<bigjools> lifeless: yeah I looked in the reports and I can only find one Unauthorized response
<StevenK> Can someone run the buildd-slavescanner.txt test on devel and see if python starts taking up gobs and gobs of RAM?
<lifeless> bigjools: how did you look - grep or something else ?
<bigjools> since I'm a buildd admin I'll play later
<bigjools> lifeless: I used my email client to search my OOPS reports emails
<lifeless> bigjools: they only report the top 25
<lifeless> by volume
<bigjools> lifeless: ok then I leave it up to you to put the OOPS on that bug :)
<lifeless> hmm, elmo linked it
<lifeless> sec
<bigjools> ah cool
<lifeless> bigjools: OOPS-1687L750 I think
<bigjools> got it, ta
<wgrant> bigjools: Do you have a few minutes to talk ddebs?
<bigjools> wgrant: not now, sorry, I'm hellish busy
<wgrant> Heh, OK.
<jml> hello
<lifeless> hello
<bigjools> hello
<jml> lifeless, regarding the deprecation warning you were seeing earlier
<jml> lifeless, I spent some time trying to fix the opacity of the message.
<jml> lifeless, and then gave up. import handlers and warn-on-import warnings are tricky beasts.
<wgrant> Is the legendary lazr.importguardian any better in this respect?
<jml> wgrant, I'd be surprised if it was.
<jml> were.
<wgrant> :(
<jml> as far as I can tell, to do it right you'd have to monkeypatch warning.warn and a couple of other methods and change the stacklevel argument
<jml> OR
<jml> just do stack introspection in the warninghandler
<wgrant> Yay.
<jml> but I'm increasingly unsure what our warning handler actually wins us.
<gmb> lifeless, So, are feature flags available to use now? I appear to have missed the news on this one.
<lifeless> gmb: yes [caveats may apply]
<gmb> lifeless, !caveats ;)
<lifeless> gmb: the caveats are that there is no UI for admining them yet, and little polish - not many scope types etc
<lifeless> gmb: but that doesn't matter much
<gmb> lifeless, Righto. Thanks. Good to know.
<lifeless> gmb: as a consumer of flags, you just write your code guarded by a flag
<lifeless> the admin stuff that is missing will affect QA and production only
<lifeless> QA to see how your patch looks on/off
<gmb> Okay.
<lifeless> production to configure the rules you request (which you can do anytime: because it defaults off you can land your patch, tweak some more, land that, and then ask losas to turn on the feature flag)
<gmb> Right
* lifeless changed the topic of #launchpad-dev to: Launchpad Development Channel | week 1 of 10.09 | PQM is OPEN | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews
<bigjools> anyone familiar with ARM know how many architecture variations there are, roughly?
<lifeless> do you mean specific SoC's ?
<bigjools> that sort of thing yeah - they each need a separate distroarchseries
<jml> lots, I reckon.
<bigjools> and I heard that our current set of supported architectures will balloon
<jml> #linaro might be a better place to ask.
<bigjools> jml: yes, that's my approx guess too :)
 * bigjools was not aware of that channel, aha
<lifeless> yes, 'lots' is definitely the answer
<lifeless> 1 or more per silicon vendor AIUI
<bigjools> the reason I ask is because people want arch checkboxes on the "register a distroseries" page but that's probably not possible
<lifeless> there is a big effort going on
<lifeless> to make the kernel and libc (IIUC) the only things that vary, so that it would be more pocket-like, or something.
<lifeless> and by vary I mean 'loadable modules'
<lifeless> jml: I haven't landed your patches yet.
<lifeless> jml: however I did release a new project :P
<jml> lifeless, I had noticed both things.
<lifeless> jml: and your patches are now very high up the pile
<jml> lifeless, grats on the new project. I'm looking forward to giving it a try.
<lifeless> I appreciate your patience and regret the time its taken to get to them
<jml> lifeless, np :)
<deryck> Morning, all
<lifeless> hi deryck
<jml> deryck, good morning
<lifeless> StevenK: you should email the list about this memory problem
<lifeless> StevenK: I'm seeing the same death on my ec2 land calls
<lifeless> so we're essentially in stop-the-line mode
<lifeless> or someone should
<lifeless> ec2 instances dying at 1744 tests run in the log, memory epxlosion and OOM killing
<lifeless> however, its mightnight, so me, I'm-a-sleeping
<lifeless> jml: tag, you're it. ^ :)
<jml> hmm ok.
<StevenK> jml: I have tracked it down, and I can share notes, if you wish
<jml> StevenK, that'd be great, thanks.
<poolie> hi
<StevenK> jml: The troublesome test is lib/lp/soyuz/doc/buildd-slavescanner.txt, the troublesome line in the test is 51, and I have a traceback: http://paste.ubuntu.com/478766/
<StevenK> jml: Evidently, I did kill the test horribly to get that traceback, so it might not help much
<jml> StevenK, what happens when you run it locally?
<jml> poolie, hello
<poolie> (just saying hi)
<StevenK> jml: It consumes gobs of memory before I get sufficently nervous and kill it
<wgrant> You know, that's interesting. It's almost the same place the hang happened over the weekend.
<jml> well, that's a good sign.
<jml> (Imagine how much worse this would be if it behaved nicely locally)
<jml> my first hunch is that it's a bug in z.testing.testrunner.
<mars> jml, + for testr failing --list
<mars> jml, any progress on the suite OOM error?  Why do you suspect zc.testing.testrunner?
<jml> StevenK, I don't see the error when running on stable.
<jml> mars, because of the traceback.
<mars> I think you hacked on that code for subunit.  At least it should be familiar territory
<jml> not the traceback formatter.
<jml> but familiar enough.
<jml> oddly enough, this interrupts me during some fairly deep surgery on ec2test/remote.py
<StevenK> jml: I've posted to -dev about it, so you can follow up when you wish
<mars> jml, StevenK, can you reproduce this on a local copy of devel?
<StevenK> mars: I can
<jml> mars, I'm trying that right now.
<jml> mars, but I always try stable first.
<StevenK> I get a lot more nervous on my desktop than a random EC2 instance
<mars> hehe
<mars> oh, awakening the OOM serial killer bit
<jml> StevenK, computers are for burning!
<mars> browsers and editors beware
<StevenK> For instance, I let Python consume 6.8GiB of RAM before the oom-killer stepped in, but on my desktop, I killed it myself after 2.1GiB
<jml> I *can* reproduce this on devel, it seems.
<StevenK> jml: Says you :-P
<jml> although please forgive me while my desktop grinds :)
<jml> this is good news!
<mars> argh - zope testrunner explicitly forbids running under PDB.  PDB would catch the original exception before formatException() got to it.
<mars> jml ?
<jml> well, it means the problem is in http://paste.ubuntu.com/478825/
<jml> and if I were a betting man, I'd say in r11345
<StevenK> I have to agree
 * jml verifies
<wgrant> .........
<wgrant> Aha.
<jml> fwiw, this is the stack trace I got when I interrupted.
<jml> http://paste.ubuntu.com/478827/
<wgrant> That would explain *everything*.
<wgrant> And why I couldn't work out what was happening over the weekend.
<wgrant> Taking that CP into account, the problem is obvious...
<wgrant> bigjools: ^^
<poolie> StevenK: you know about ulimit?
<jml> ... and science verifies it is indeed that revision.
<jml> wgrant, it's not obvious to me (other than "signals are hard")
<mars> heh
<wgrant> jml: Note that one of the cases results in an infinite loop.
<wgrant> It might not be the problem that's causing the excessive memory usage. But it is what caused the crisis over the weekend.
 * jml looks atthe diff
<wgrant> If you stick a try/except in a 'while True' loop, you want to ensure that all the paths have a way to escape...
<jml> I seem to remember recommending not to use an infinite loop.
<StevenK> poolie: I do
 * bigjools did not use an infinite loop
<jml> ahh.
<wgrant> bigjools: Why is there an infinite loop, then?
<bigjools> fuck nose
<wgrant> Heh.
<jml> bigjools, I'm looking at the code. It's clear that the intent is not to be an infinite loop
<bigjools> has this revision passed buildbot yet?
<jml> but you know what they say about good intentions.
<jml> bigjools, no.
<bigjools> ooookay then
<bigjools> that explains the buildbot failures
<jml> so, the memory thing is because builder.failBuilder(str(reason)) is appending stuff to a list?
<bigjools> sigh
<bigjools> so, where's the loop coming from
<wgrant> I don't think it appends stuff to a list.
<wgrant> bigjools: The codepath that was problematic over the weekend.
<wgrant> After calling failBuilder... it logs, then returns to the top of the loop.
<bigjools> wgrant: that is now obvious
<mars> bigjools, ah, this is why two buildbot slaves fell over from Friday onwards, eh?
<bigjools> but how is it returning to the top of the loop
<wgrant> bigjools: How not?
<jml> bigjools, it just continues after the except
<bigjools> there's a "return"
<wgrant> bigjools: Only the second except and the else have a return.
<jml> bigjools, not in the failBuilder clause.
<bigjools> ooooooooooooooo ffffffffffffffuuuuuuuuuuuuuuuuuuuuuck
<wgrant> Haha.
<jml> fwiw, the loop could be written as 'for i in range(MAX_EINTR_RETRIES)'
<bigjools> jml: please feel free to do that.  I tried and the code became less readable.
<jml> bigjools, I'll have a stab.
<bigjools> cheers
<bigjools> jml: in the meantime I'll land a fix
<wgrant> This explains why a look at db-devel yielded no reasonable explanation for the weekend's behaviour.
<bigjools> yes :/
<bigjools> god DAMN
<wgrant> jml: It's possible that the logging goes to a StringIO... that's probably likely, in fact.
<mars> bigjools, so you will have to land a roll-back revision before we can restart the builders then
<jml> wgrant, that would make sense.
<bigjools> mars: eh?
<mars> bigjools, I assume the tests in the revision caused the OOM errors?  And the OOM killed our CI build farm.  So the revision needs to be backed out.
<jml> bigjools, http://paste.ubuntu.com/478837/
<bigjools> mars: why can't I just submit a fix?
<mars> bigjools, or that, if you think it is fast to do
<bigjools> I do!
<bigjools> jml: thanks
<jml> bigjools, I can land that now, if you'd like.
<jml> (it also has the return fix)
<bigjools> jml: I'll do it
<bigjools> I need to CP this branch as well
<jml> ok
<bigjools> easier if I do the whole thing
<jml> no arguments from me :)
<wgrant> Is there a good reason not to just have a single return at the end?
<jml> wgrant, I reckon that would be better, yes.
<wgrant> Slightly less explicit. But also slightly less likely to destroy everything.
<mars> depends how close you want to keep the returns to the code that depends on them
<mars> for readability
<jml> mars, the readability issue here is that the control flow is jumpy
<mars> yep
<bigjools> so I can see exactly what happened on Saturday now
<bigjools> we got a xmlrpclib.Fault when an Enablement machine was pulled
<bigjools> and the loop ensued :/
<jml> bigjools, fwiw, the test passes locally with the fix.
<wgrant> It's somewhat more complicated than that, I think. Since it tried to abort.
<bigjools> yeah
<wgrant> But that's the main bit of it.
<bigjools> yup
<wgrant> Now I don't have to go insane wondering what I missed :)
<jml> perhaps ec2 test should run against stable by default, rather than devel.
<bigjools> jml: http://pastebin.ubuntu.com/478840/
<mars> salgado, don't worry about deactivating the Lucid EC2 AMI.  The OOM test failures were unrelated to the Lucid upgrade.
<bigjools> jml: however with that change, test_updateBuilderStatus_catches_repeated_EINTR fails
<salgado> mars, yeah, just saw the backlog. :)
<jml> meh. I shouldn't have been so quick to delete my branch with that change :)
<bigjools> jml: I see why.  This is why I didn't use range() :)
<jml> bigjools, why?
<bigjools> jml: it's not running the code at the bottom of the exception any more
<bigjools> except in the case of reason[0] != errno.EINTR:
<jml> bigjools, the bottom of which exception?
<bigjools> except socket.error
<jml> oh, you mean in the case where MAX_EINTR_RETRIES is actually reached
<bigjools> yes
 * jml tries a fix
<jml> http://pastebin.ubuntu.com/478842/ works
<bigjools> jml: do you see my point about it being less readable now?
<jml> but maybe http://pastebin.ubuntu.com/478843/ really is the patch with the best cleanness / robustness trade-off.
<bigjools> jml: that has the same problem
<jml> bigjools, there are so many problems, which one do you mean?
<bigjools> jml: the one where it doesn't run handleTimeout() when we hit MAX_EINTR_RETRIES
<jml> bigjools, it does. run the tests :)
<bigjools> jml: oh sorry I can't read
<bigjools> jml: ok I'll land that with your blessing?
<jml> bigjools, please.
<bigjools> and we need to cowboy cesium until the CP goes in
<jml> last iteration: http://pastebin.ubuntu.com/478848/
<bigjools> jml: land that afterwards if you wouldn't mind, I'm already mid-process for the other change
<bigjools> ok buildd-manager was restarted with the patch
<jml> bigjools, sure thing.
<barry> EdwinGrubbs: btw, whatever happened with that ml oops?
<EdwinGrubbs> barry: well, it was just spam, so I am now looking at adding the full text of the email to the oops, so that we don't have to track down a losa to see it.
<barry> EdwinGrubbs: +1.  i also suggest that we track down that rt and try to work with IS to improve the incoming mta (exim i believe) anit-spam defenses.  really, stuff like this should rarely if ever actually hit lp
<barry> s/anit/anti/
<EdwinGrubbs> barry: I tried searching rt for requests concerning spam or filtering for mailman, but no luck. Do you have any other information before I create a new request?
<barry> EdwinGrubbs: unfortunately no.  i'm sure i no longer have a link to the rt.  probably can't hurt to just open a new issue
<EdwinGrubbs> ok
<mars> bigjools, jml, I see your change landed on devel - so are we good to restart the buildbot farm?
<bigjools> mars: yep, thanks
 * bigjools dons brown paper bag
<mars> losa ping, could we please restart the buildmaster and the downed slaves?
<mars> losas, I have no idea what state the lucid buildslaves are in - they probably need a process tree cleanup :/
<jml> mars, fwiw, I've split TestOnMergeRunner into three different classes
<mthaddon> mars: erm, you mean lpbuildbot (buildmaster is the PPA/archive build master)
<mars> jml, wow, 3?
<mars> mthaddon, yep!  Thanks
<sinzui> jml, mumble?
<jml> mars, yeah. one to handle running generic stuff in a daemon; one as an object that represents the merge request; one as the thing that knows how to run tests and gather results
<jml> that middle object simplified a lot of stuff, I think.
<jml> sinzui, sure.
<mars> mthaddon, the lpbuildbot farm has a buildmaster as well - wonderful confusion :)
<bigjools> mars: I often get confused by this nomenclature
<bigjools> I clicked on a graph earlier on and it took me 10 seconds to realised why the thing didn't make sense
<mars> lol
<mars> we should just call it the bbmaster or something
<mars> bbmaster, bbslaves
<EdwinGrubbs> jelmer: ping
<jelmer> EdwinGrubbs, pong
<EdwinGrubbs> jelmer: I was wondering if you had a chance to use the feature on edge that lets you create project from a source package. I was starting to QA it but I see there are a lot of meta packages and packages where ubuntu is the upstream, so they don't need a project.
<EdwinGrubbs> jelmer: I'm also curious, what is the biggest benefit you get after you link a project to a package?
<jelmer> EdwinGrubbs: Because of the way the sorting in the needs-packaging list works all of the packages that can be linked at the beginning of the list already have been, that's why there are so many native packages there.
<jelmer> If you skip a few pages in the list there should be some more packages that don't have an upstream project registered.
 * jelmer looks for one
<jelmer> e.g. https://edge.launchpad.net/ubuntu/maverick/+source/soprano
<jml> sinzui, http://www.jjg.net/elements/pdf/elements.pdf
<EdwinGrubbs> jelmer: what's the first thing you do with a project after it has been linked to take advantage of it?
<jelmer> EdwinGrubbs: usually I register the upstream branch, and sometimes I add the homepage.
<EdwinGrubbs> jelmer: and do you use the upstream branch to automate some of the package building process?
<jelmer> EdwinGrubbs: not necessarily, usually it's just so I can get a local copy of the upstream source code or perhaps work on a patch against upstream
<EdwinGrubbs> jelmer: so, it just makes it easier to get the project in bzr, instead of switching to git or svn to work on a patch? Sorry if this is a lot of questions. I'm trying to improve my understanding of the process.
<jelmer> EdwinGrubbs: Yep, exactly. It gives me an easy way to track all of my in-progress patches (since they're all nicely listed on my branches page) independent of what project it's for.
<EdwinGrubbs> ahhh
<jelmer> EdwinGrubbs: No problem; this is just how I use it though, I imagine it's quite different for other people.
<EdwinGrubbs> jelmer: who else would be good to talk to?
<jelmer> EdwinGrubbs: Specifically in relation to linking upstream projects and Ubuntu source packages?
<EdwinGrubbs> jelmer: yes
<jelmer> EdwinGrubbs: Ubuntu developers that use bug watches (as registering the upstream bug tracker is required to do bug watching IIUC) and forward bugs from ubuntu to upstream.
<EdwinGrubbs> ok
<jelmer> EdwinGrubbs: The early adopters of daily build packages might also be good candidates, as they need both the upstream source branches and ubuntu packaging branches to build their packages.
<jelmer> I'm not sure if the actual link between the upstream and the ubuntu source package is of benefit to them though, just that they use both the upstream project and the packaging in ubuntu.
<Ursinha> sinzui, hi, is the partial fix for bug 607879 really testable, or should we mark it as qa-untestable for now?
<_mup_> Bug #607879: ~person/+participation timeouts <qa-ok> <timeout> <Launchpad Registry:In Progress by jcsackett> <https://launchpad.net/bugs/607879>
<sinzui> Ursinha, it only reduced the query count
<Ursinha> sinzui, so not really testable
<sinzui> Yes, We can only see query counts go down.
<EdwinGrubbs> jelmer: thanks for the info
<Ursinha> sinzui, I'll change the bug tag then. Thanks
<sinzui> thanks
<EdwinGrubbs> rockstar: does sourceforge let us mirror their branches? I remember this being a problem a long time ago, but I can't remember if it was resolved.
<rockstar> EdwinGrubbs, I believe so.
 * jml hates writing tests after writing the code
<jelmer> EdwinGrubbs: I think in the case of irda-utils you probably wanted to import /trunk rather than the root of the repository.
<sinzui> benji, ping
<sinzui> benji (or gary_poster), jtv has been working on a fake librarian for testing.  He registers it using zope.component.provideUtility, but (simply by unregistering the utility it provides) fails to restore the original ILibraryFileAliasSet utility
<sinzui> I got stuck on this a few months ago and change my implementation since I could not solve the deregistration issue
<gary_poster> jtv was just trying to talk to me about this when his connection died
<gary_poster> sinzui, talking to jtv about it in private message
<jtv> sinzui: thanksâ¦ sorry for dropping out there; pidgin died
<EdwinGrubbs> jelmer: oops
<lifeless> moin
<lifeless> is the issue StevenK reported with ec2 runs OOMing fixed ?
<salgado> lifeless, yes
<lifeless> has it landed in devel - if I just ec2 land, will my stuff go through (assuming my bits are good)
<salgado> yes, mine just went thrrough
<lifeless> \o/
<lifeless> time to fire up the engines boys!
<jml> hello what
<jml> lifeless, the fix has yet to reach stable, according to lpbuildbot.
<jml> I have just added a new option: "ec2 test --dont"
<lifeless> jml: long as its in devel
<mars> jml, 'ec2 test --dont' ?
<jml> mars, yeah. it sets up the instance ready to run the test suite, and then it doesn't.
<mars> jml, heh, I've been instructing people to run 'ec2 demo'
<jml> mars, yeah, but that does more stuff.
<mars> or "ec2 test -o '-t somejunk'"
<mars> "ec2 test --postmortem -o '-t somejunk'"
<mars> that leaves the instance running
<jml> mars, also, it doesn't conveniently tell you the command that ec2 would run if it were going to run the tests.
<mars> true
<mars> jml, ec2 test --not-really  :)
<jml> yeah. I might rename it.
<mars> --setup-only works too
<mars> jml, in your list mail about the OOM problem, what do you mean by "Test suite failures often really do mean production failures" ?
<mars> no one has mentioned a cowboy or production issues in the thread
<jml> mars, oh. well, there was both :)
<jml> the sequence went something like:
<jml>   * critical production issue A
<jml>   * cowboy fix for issue A
<jml>  * land fix for issue A
<jml>  * critical production issue B caused by fix for issue A; critical test suite failure caused by fix for issue A
<jml> and then you can pick up the rest from there.
<mars> nice
<mars> impressive - that one block of code knocked out production and our development pipeline
<mars> I don't think that has happened before
<Ursinha> rockstar, hi, is https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1686EA3556 really supposed to be an oops?
<rockstar> Ursinha, yes.  If we're not catching it, then there's a bug there.
<Ursinha> rockstar, I mean, is that supposed to fail as an oops or just be caught to display a nice error message to the user?
<rockstar> Ursinha, the latter.  The bug is that we're not catching it.
<Ursinha> rockstar, right. mind if I file a bug for that?
<jml> mars, well, by far the most common pattern is to run tests before applying a change to production.
<mars> jml, heh, I was just writing a reply asking about that.  So why did we not do so this time?
<mars> jml, and could we have just run '-m lp.soyuz' and felt confident enough with that?
<jml> mars, I don't know exactly. If I had to guess I'd say that it would have taken too long.
<jml> mars, I'm not 100% sure that would have caught the failure.
<mars> I am just wondering if there is a natural suite partition there
<jml> mars, do you mean as code currently stands now, or as it ought to be?
<mars> as it ought to be
<jml> mars, probably.
<mars> there is no way to run 'soyuz-not-app-server-code', or 'soyuz-just-the-app-server', but we can run 'soyuz'
<mars> which should be much faster that 'soyuz+bugs+code+..."
<mars> I'll try timing it, see what happens
<jml> mars, I'm not sure if you know, but lots of the code that runs that particular production system lives in paths that do not have 'soyuz' in them at all.
<mars> jml, I did not know that
<jml> lp.buildmaster being the one that I'm most certain about.
<mars> yes, was just looking at that
<jml> dammit. testing this refactoring shows that I am basically incapable of writing Python without unit tests.
<lifeless> jml: :)
<jml> I just fixed my fourth simple name error.
<mars> jml, pyflakes?
<jml> mars, doesn't help with instance variables.
<lifeless> jml: interestingly I suggested a range() approach too, back when reviewing the EINTR change
<jml> lifeless, well, a little paranoia is healthy, and all that
<jml> lifeless, I notice that Twisted & bzrlib both have until_no_eintr-style helpers and that both use infinite loops
<lifeless> yeah
<lifeless> I haven't read your mail yet
<jml> I grant that the likelihood of a call generating eintr forever is pretty small
<lifeless> in a call with the isd guys about logging & stuff
<lifeless> jml: lamont argues that forever is the right answer
<jml> but something in me bristles all the same
<jml> lifeless, for the moment, I disagree. Everything needs a circuit breaker.
<jml> also, it somehow became 8pm.
<jml> one more test run...
<lifeless> jml: I suggested that perhaps the buildd manager become an API client
<lifeless> jml: bigjools felt that this was a terrible idea
<jml> lifeless, I don't think it's a terrible idea, but I'm not sure how it would help.
<jml> lifeless, or rather, the first thing that needs to happen to the buildd manager is to clean up a bunch of needlessly complex code.
<lifeless> jml: it would have meant that there wasn't a deadlock on the builder row in the DB, so the disabling would have worked; the b-m might still have broken, but less cascade would have happened
<lifeless> I worry about the thread-locals model of zope with the context model of twisted
<jml> lifeless, oh huh.
<jml> lifeless, we managed to get it quite stable with the old authserver
<jml> lifeless, but it was a trial.
<jml> lifeless, switching to internal xmlrpc helped a lot.
<lifeless> perhaps its only a worry
<lifeless> but I would be happier with the scheduler being just a thing that talks to webservices and local processes
<jml> two thoughts
<jml> 1. there are no good twisted clients for the API
<lifeless> didn't jamesw make one?
<jml> lifeless, it's a prototype
<lifeless> <wry>aren't they all?</wry>
<jml> 2. having some kind of deferred-returning calls for the db stuff makes integration points _really_ obvious, which is nice.
<lifeless> one of the problems is this
<lifeless> model objects know they are in a transaction
<lifeless> but we don't want long lived transactions
<lifeless> anyhow
 * jml is off.
<jml> g'night.
<lifeless> gnight
* lifeless changed the topic of #launchpad-dev to: Launchpad Development Channel | Performance tuesday! | Week 1 of 10.09 | PQM is OPEN | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews
<Ursinha> sinzui, I see that bug 612408 is marked as Fix Released but I still see occurrences of that OOPS on lpnet
<sinzui> It is not fix released...
<sinzui> damn it, the branch was marked fix committed when we knew we needed several branches to fix this.
<sinzui> Ursinha, it is in progress. We are waiting for jcsackett's branch to land
<sinzui> I updated the status and milestone
<Ursinha> sinzui, right. So, if you want to avoid QA bot to change this bug, just add the [incr] tag to your commit msg, or if using ec2 land, there's the --incremental option
<Ursinha> sinzui, so it won't close your bug until you land a fix without the incremental tag
<jcsackett> Ursinha: i think that's info more directed at me, and my apologies if i screwed up QA process.
<lifeless> deryck: sorry about the wrong project, the back-link seemed to leave project-wide bug filing state all confused :)
<jcsackett> Ursinha: thanks for mentioned the incr tag.
<Ursinha> jcsackett, ah, no, my fault I haven't announced that properly, I'm afraid
<deryck> lifeless, dude, no worries.  Just doing CHR duties today.
<jcsackett> Ursinha: either way, thanks for the info. :-)
<Ursinha> jcsackett, my pleasure :)
<lifeless> deryck: have we had any feedback on +filebug ?
<lifeless> anyone come and loved us or hated us ?
<deryck> lifeless, not that I know of.  No feedback at all.  Neither for or against.
<deryck> no news is good news, I hope.
<lifeless> well thats something
<lifeless> certainly no noose is good nes
<lifeless> (gotta love mel brooks)
<Ursinha> sinzui, did you update the bug? I can't see it here...
<Ursinha> ah, just showed up
<Ursinha> sinzui, jcsackett, thanks
<sinzui> Ursinha, https://bugs.edge.launchpad.net/launchpad-registry/+bug/612408
<Ursinha> sinzui, thanks, it just updated here. don't know why it took so long, though
<sinzui> I see In progress, 10.09, oops qa-bad
<Ursinha> sinzui, now I see the same  too
<lifeless> Ursinha: farming the oops reports ?
<Ursinha> lifeless, yes sir
<lifeless> Ursinha: how are we placed for the merge workflow ?
<lifeless> Ursinha: anything I can help with ?
<lifeless> which reminds me, time to check the rt ticket status
<Ursinha> lifeless, sure, you can take the rollback part if you want. I had no time to tackle that last week, am working on writing more tests to it
<lifeless> I won't today, because its performance day, but lets talk about that ~ this time tomorrow, and I'll likely make the same offer again :- but actually do it :)
<Ursinha> lifeless, thanks
<lifeless> we're only waiting on the tagger and rt 40482:live-schema staging environment
<lifeless> AFAIK
<lifeless> flacoste: hi
<flacoste> hi lifeless
<lifeless> we're on, I think ?
<flacoste> lifeless: we are!
<deryck> Later on, all.
<lifeless> sinzui: ping
<lifeless> sinzui: I need a slow milestones url page or bug # please :)
<lifeless> flacoste says landscape/+milestones ?
<sinzui> lifeless, yes, that is often a good one *if* you can see all their private bugs
<sinzui> project groups are a little different that projects and distros.
<lifeless> flacoste: https://devpad.canonical.com/~stub/ppr/lpnet/latest-daily-timeout-candidates.html
<lifeless> sinzui: I will arrange to be able to :)
<lifeless> jkakar: ping
<mwhudson> morning
<flacoste> lifeless: https://wiki.canonical.com/Launchpad/Sprints/BugJamDecember2010
<lifeless> sinzui: do you have an oops report for the landscape one ?
<lifeless> sinzui: or a bug number ?
<jkakar> lifeless: Hiya.
<lifeless> sinzui: performance day, flacoste thought that looking at this pain point might make you happy :)
<lifeless> jkakar: hiya
<lifeless> jkakar: going to be doing some performance work on milestone views, and apparently the landscape private bugs are a contributing factor in its pain-level
<mwhudson> flacoste: 2010-12-13 to 2010-12-24 is a bit more than a week
<lifeless> jkakar: but I'd need to be able to see them to see the issues
<lifeless> jkakar: so, I was wondering, if I'm not already, if I could be in the relevant group for a week or two
<jkakar> lifeless: I'm happy to add you to the landscape team, but unfortunately you'll get a lot of mail.
<lifeless> jkakar: thats ok, I'll treat it the same way I treat launchpad bugs :)
<jkakar> lifeless: Added.
<lifeless> thanks
<jkakar> lifeless: And by "same way I treat launchpad bugs" that means we should expect merge proposals from you, right? ;b
<lifeless> perhaps...
<jkakar> Hehe
<lifeless> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1689EB3060 is a sample from https://edge.launchpad.net/landscape-project/+milestones
<lifeless> 305 16ms statments
<lifeless> 190.0ms for the longest statement
<lifeless> I think I can do something for this
<poolie> hello, happy performance day
<lifeless> thank you
<sinzui> lifeless, there is no current bug number for the Milestone timeout. It reappeared last week after the cache was removed.
<EdwinGrubbs> matsubara-afk: ping
<sinzui> lifeless, this is the bug that was tracking milestones with lots of private bugs: https://bugs.edge.launchpad.net/launchpad-registry/+bug/447418
<sinzui> ^ it was closed after the oopses disappeared. I had changed the security on the bugs to lp.View after verify them once.
<lifeless> sinzui: thanks
<sinzui> lifeless, this is also the bug where I discovered the number of assigned users is also a factor. We format links to them. Ubuntu has a lot of assignees :(
<lifeless> sinzui: I will look, journal what I find, and see if I can help
<sinzui> thanks
<lifeless> would you like me to edit that bug
<lifeless> or make a new one ?
<sinzui> Lets reopen it since it has made a few occurrences this week
<lifeless> ok, I'll do so
<sinzui> lifeless, was pondering creating a memcached rule for person/pillar link formatters. 250 assignees is a lot of icon lookups
<lifeless> damn, a failur ein cacheproperty branch >< ah well, ec2 knows best :)
<lifeless> sinzui: What does an icon lookup entail ?
<sinzui> librarian calls
<lifeless> wha?
<lifeless> we render them on the appserver?
<sinzui> we look for an icon, then insert a link to the librarian icon for the link
<lifeless> thats just a query then
<sinzui> yes
<lifeless> any reason we can't prepopulate it ?
<sinzui> Since we link to users on every page, should all assignees, bug/branch/question commenters be prepopulated?
<lifeless> I suspect so
<sinzui> only teams have icons, only teams can be private (not rendered). Teams do not change their icons very often.
<lifeless> sure
<lifeless> at the lowest level though, a lookup is a lookup, and postgresql is as good as memcache at doing those very quickly
<sinzui> Since we would need a separate memchache mechanism, we could build once with a knowable key that we can invalidate on change
<lifeless> but unlike memcache we can do them all at once, whereas with memcache we have a serialised interface (due to our appserver structure)
<lifeless> so it will be faster to do this in postgresql
<lifeless> (I think)
<wgrant> memcached is really slow when making lots of requests.
<sinzui> oh, that reminds me. Last time I looked at the bugtask badge decoration code, we were looking for a mentoring icon. We should stop that
<lifeless> yeah this page is definitely death by sql
<sinzui> It is faster than lots of pg requests I am told
<lifeless> SQL time: 7505 ms
<lifeless> Non-sql time: 7849 ms
<lifeless> Total time: 15354 ms
<lifeless> Statement Count: 390
<wgrant> sinzui: Yes, but so is a snail.
<lifeless> sinzui: single-pg-request < many memcache requests < many pgsql requests
<sinzui> I have not yet set my email outlining the death by many menus I am see hiding in our oopese
<lifeless> ooh, interesting
<lifeless> so wgrant, going to join in performance tuesday today ?
<wgrant> If beat-my-project-team-into-doing-work doesn't take up too much time.
<lifeless> wgrant: \o/
<lifeless> wgrant: I can fedex a bat from a local friend, if needed ?
<wgrant> Did the buildd-manager stuff get sorted out last night?
<wgrant> Heh.
<lifeless> yes
<sinzui> lifeless, I need to start a dinner. I promised my team I would send an email outlining to cost of using the existing menu implementation with a suggest of how to address it.
<lifeless> sinzui: start your dinner then :)
<wgrant> I'm a little concerned that a rev seems to have been CPd without making it through the test suite. But I guess it was urgent enough that it may have been cowboyed.
<lifeless> wgrant: it was
<wgrant> Ah, good.
<lifeless> so, https://edge.launchpad.net/launchpad-foundations/+milestone/10.08 seems to be a regular milestone page with all the features present
<lifeless> does create_initialized_view process the template etc?
<thumper> lifeless: what do you mean?
<thumper> lifeless: it creates the view and initializes it
<wgrant> Can someone check buildd-manager logs? It's been stalled for a few minutes.
<lifeless> I don't know what 'initializes it' entails
<lifeless> I want to make sure that I capture all the SQL done by the view & template rendering
<lifeless> whats the most tasteful way to do that.
<thumper> lifeless: it calls the initialize method
<thumper> lifeless: which sets up any fields and widgets
<thumper> lifeless: it does not render the view
<lifeless> ok
<lifeless> so what should I do accomplish my goal
<thumper> lifeless: that is either __call__ or render
<thumper> not sure which is the best way
<thumper> probably to use a browser test case
<thumper> and get the test browser to load the page
<lifeless> ok
<wgrant> So, we now have two or three untested buildd-manager CPs, and it appears to have fallen over.
<lifeless> losa ping
<wgrant> Well, someone could check the logs.
<lifeless> yes
<lifeless> someone experienced, who knows that bit, and who can act to fix it, would be best, no ?
<wgrant> True, but LOSAs are unlikely...
<wgrant> Oh, I guess some might be back.
<lifeless> I know some are back :)
<wgrant> Well, apparently not.
<lifeless> wgrant: I have one in a private channel - sorry. But they are feeling flat-out - multiple issues at once etc
<wgrant> Ah.
<mbarnett> wgrant: trying to figure out what is going on with the builders
<mbarnett> wgrant: i was led to believe you might have a working theory?
<wgrant> mbarnett: No idea. What are the logs saying?
<mbarnett> wgrant: still poking around.  last i saw was a "no route to host error"
<mbarnett> trying to get more info now
<mbarnett> i think the buildd-master may be well and truly hung
<mbarnett> it is logging nothing, and the running process doesn't seems to be actually doing a single thing.
<mbarnett> Gonna give it another minute and see if it wakes up.  If not, it will be time to hit it with things.
<wgrant> lifeless: What's the best course of action to debug that? strace, then SIGINT and hope we get a traceback?
<lifeless> strace
<lifeless> well
<lifeless> ps first
<lifeless> see what state the process is in
<lifeless> then strace, which may 'fix' it
<mbarnett> strace gives NOTHING
<lifeless> if that doesn't, attach gdb and get a backtrace
<mbarnett> lp_buildd@cesium:/srv/launchpad.net/production-logs$ strace -p 10680
<mbarnett> Process 10680 attached - interrupt to quit
<mbarnett> select(39, [9 38], [], [], NULL
<lifeless> that probably means its dropped the ball on a deferred
<mbarnett> there was an error in the logs when the process hung
<wgrant> :( All the CPs are tested.
<mbarnett> "no route to host"
<wgrant> mbarnett: What was the full line?
<mbarnett> wgrant: the full line from the strace?
<wgrant> mbarnett: From the 'no route to host' line.
<mbarnett> http://pastebin.ubuntu.com/479111/
<mbarnett> wgrant: ^
<wgrant> Well.,
<wgrant> This is the same codepath that died last time.
<wgrant> There's nothing after that log entry?
<mbarnett> awesomesauce
<mbarnett> nope
<mbarnett> that's it
<wgrant> WTF.
<wgrant> Does that still have the two cowboys, or is it properly CPd now?
<mbarnett> CP'd
<mbarnett> actually, let me verify
<wgrant> So, we have replaced an infinite loop with a hang. Awesome.
<mbarnett> nope, it is cowboy'd
<mbarnett> with http://pastebin.ubuntu.com/478843/
<wgrant> That's the only cowboy?
<mbarnett> there is another that is not listed on the production status page
<wgrant> Should be to the same piece of code.
 * wgrant tries to reproduce locally.
<mbarnett> http://pastebin.ubuntu.com/479112/
<mbarnett> that is also on there
 * mbarnett goes to update the production status page
<wgrant> Oh, wait, it isn't quite the same codepath as last time. But it's right next to it.
<thumper> \o/
<wgrant> Hm, there's only a test diff?
<thumper> it isn't the code tests breaking the translation tests
<wgrant> Nothing in lib/lp/buildmaster/model/builder.py itself?
 * thumper confirms by running the other half
<mwhudson> i completely don't see how a failure there should hang the buildd-manager
<mwhudson> it might stop that builder from working forever more
<mbarnett> wgrant: 478843 is a cowboy to builder.py
<wgrant> 478843?
<mwhudson> wgrant: http://pastebin.ubuntu.com/478843/
<wgrant> mwhudson: Of course it shouldn't.
<wgrant> Oh.
<wgrant> But this is buildd-manager.
<wgrant> Well, I can't see what's going on. May be good to gdb out a backtrace.
<wgrant> (and this time I think I'm actually taking all the cowboys into account...)
#launchpad-dev 2010-08-17
<mbarnett> bah, gdb not installed
<wgrant> rofl
<mwhudson> wgrant: probably, but i suspect that it will be very boring, just twisted sitting in an select loop with no active sockets
<lifeless> a clean shutdown *should* tell us about un-gc'd deferreds
<lifeless> also
<lifeless> triggering a gc from gdb should do that
<wgrant> mwhudson: There should be lots of them, though...
<mwhudson> yeah
<lifeless> lsof output too then
<wgrant> Aha.
<wgrant> Got it.
<wgrant> mbarnett: Can you check if there's a running PPA builder reset trigger?
<wgrant> I'm not sure what it's called.
<wgrant> vm_resume_command is the config key.
<poolie> lifeless: flacoste suggested i should also land the flags ui into lp devel
<poolie> since at the moment it seems to now be only in db-devel
<poolie> but its precondition db changes should now be in devel
<lifeless> poolie: yes, definitely
<mbarnett> wgrant: i don't see one off hand
<wgrant> Hm, now it won't respond to SIGTERM.
 * wgrant is trying a few things locally.
<wgrant> mbarnett: There really should be one running.
<lifeless> wgrant: a hung one ?
<wgrant> lifeless: Ideally.
<wgrant> It looks like it's an ssh process.
<wgrant> ./ftpmaster/launchpad-lazr.conf:vm_resume_command: ssh -i /home/lp_buildd/.ssh/ppa-reset-builder ppa@%(vm_host)s
<wgrant> It will hang if that does, but recovers if it dies, AFAICT.
<mbarnett> here is everything running as the lp_buildd user:  http://pastebin.ubuntu.com/479127/
<wgrant> Mmm. Zombies.
<wgrant> So, it will hang until the resume trigger exits after a communication failure, bypassing even the normal timeout. But I don't see why it would continue to hang if there's no process running.
<mbarnett> hah, i think lamont came along and kicked the hung process.
<wgrant> Ah, yes, everything's running again.
<mbarnett> yeah, now we are going
<wgrant> So that was probably it.
<mbarnett> lamont> or a child ssh process killed.  /me did that, seems happier now
<wgrant> Phew.
<thumper> lifeless: ping
<mbarnett> okay, i see the ssh process he killed
<mbarnett> good to know that will get things moving agian
<thumper> lifeless: I've got the bzr failure down to two test files
<lifeless> COOL
<lifeless> bah
<thumper> lifeless: is it worthwhile trying to eliminate individual tests?
<lifeless> Cool
<lifeless> thumper: it may well be
<thumper> lifeless: ok
<mbarnett> well, all builders are now actually doing stuff, so that is good
<wgrant> mbarnett: Bug #618955
<wgrant> Bug #618955?
<wgrant> :(
<mbarnett> thanks for filing that.  I have subscribed us losas
<thumper> lifeless: gc.collect likely to screw things up?
<lifeless> thumper: I wouldn't expect it too
<thumper> lp.translations.scripts.tests.test_message_sharing_migration.TestSharingMigrationPerformance
<thumper> lifeless: that test makes the other one fail
<thumper> lifeless: I'm just seeing if it is either test in that test case, or a specific one
<lifeless> progress!
<thumper> lifeless: either of the two tests in that testcase cause the failure
<thumper> lifeless: calling TestSharingMigrationPerformance._listLoadedObjects triggers the error
<thumper> which uses gc.get_objects
<thumper> I'm guessing that is it
<lifeless> zomg
<lifeless> yes, a gc held reference to a lazy object is equivant to a bound alias
<lifeless> don't-do-that
<lifeless> :)
<thumper> advice needed on how to fix this test
<lifeless> hmm
<lifeless> easiest not to call that damn thing at all
<lifeless> its a high cost function
<lifeless> perhaps installing a Storm Tracer
<lifeless> its also a poor surrogate for what I think they want to test
<thumper> I don't really understand their test
<lifeless> so
<lifeless> I think what they want to say is
<lifeless> 'if there are 15 pomsgids in memory before you call this function, there are only those 15 in memory afterwards'
<lifeless> and the implication is that no pomsgids were loaded in the interim
<lifeless> this only works if there is a non-weak cache installed (we have one) in the storm layer, and nothing resets it or otherwise invalidates it.
<lifeless> try this
<lifeless> do a gc.collect() at the top of _listLoadedObjects
<lifeless> this doesn't invalidate the test because:
<lifeless>  - gc collect can happen anytime
<lifeless> this should fix the problem because:
<lifeless>  - we don't have any references to the scope replacer
<thumper> trying
<lifeless> but also please file a tech debt bug - this is a very roundabout way to say that something didn't happen - its a surrogate for something we can measure directly with only a little more effort
<thumper> nope
<thumper> still fails
<lifeless> darn
<lifeless> oh
<lifeless> by bad
<thumper> we have code for storm tracers
<lifeless> so that isn't working because
<thumper> if it will only take a little more to fix, I could do that now
<lifeless> there is a module in bzrlib that hasn't triggered its lazy load.
<lifeless> and we go past it twice, or something
<lifeless> though I am a little surprised it didn't work. Does it blow up in that function, or in your test code?
<thumper> we have StormStatementRecorder
<thumper> lifeless: it blows up in the following test
<lifeless> I would cjec that there are no Selects that return a <type>
<lifeless> *check*
<lifeless> that seems much more targeted, to me.
<thumper> hmm...
<thumper> the statement recorder can't check that
<thumper> it records the raw sql
<lifeless> SELECT.*POSMSGSETID\..*FROM -> boom
<thumper> I'm recording the statements now, with a pdb set
<thumper> to look
<lifeless> well
<lifeless> we don't expect to see that, because the test 'passes' :)
<lifeless> Ursinha: timeout bugs get high + triaged :)
<lifeless> zero-oops-policy
<lifeless> just fyi
<thumper> you don't see POSMSGSETID.* in the SQL
<lifeless> good
<thumper> it gets expanded to all columns
<thumper> storm expands the *
<lifeless> right
<lifeless> and it qualifies them
<lifeless> posmsgsetid.foo
<thumper> ah, I see what you mean now
<lifeless> is what you'd see when pomsgsetid's are looked up
<Ursinha> lifeless, cool, should I triage them as such when filing them?
<lifeless> Ursinha: where there is as clear a policy as the zero-oops-one, I absolutely think so
<Ursinha> cool
<lifeless> Ursinha: where its going to be team-determined, then no - or at least, I don't in those cases except
<Ursinha> lifeless, what's the difference
<lifeless> where I am putting my TA hat on and asking as a favour-to-the-TA to do something
<lifeless> Ursinha: well a non-oops non-timeout bug doesn't have a predetermined importance
<thumper> is any in python 2.5?
<Ursinha> lifeless, ah, I was only talking about timeout bugs :)
<lifeless> Ursinha: :)
<lifeless> Ursinha: I was giving a general answer :)
<lifeless> sorry for de confusion
<Ursinha> lifeless, no problem, thanks for the information :)
<thumper> lifeless: can you check out this? http://pastebin.ubuntu.com/479160/
<thumper> lifeless: do you think it is testing enough of the same thing?
<lifeless> +1
<lifeless> I certainly do
<lifeless> should be faster too
<thumper> lifeless: I'll break it into a separate branch
<thumper> lifeless: and no bad interaction
<mwhudson> lifeless: about performance tuesday, is the looking up of branch subscriptions because of the checking for launchpad.View ?
<mwhudson> lifeless: if so, have you seen how the branchlistings avoid this particular death by sql?
 * thumper files a bug on rosetta
<lifeless> mwhudson: I don't know
<lifeless> mwhudson: I'm trying to get a test that is precisely the same
<mwhudson> ok
<lifeless> mwhudson: for now, I have one that shows 'one more query as you add a private bug'
<lifeless> let me paste
<lifeless> so, I can fix one cause of too many queries
<lifeless> but I know I'll be missing the particular one :(
<lifeless> http://pastebin.com/wPNqcuzn is my test
<lifeless> mwhudson: AIUI you retrieve too much and then do a custom security check
<mwhudson> lifeless: 'you'?
<lifeless> 'branhc listings'
<mwhudson> that's the launchpad default usually, it's not what branch listings do
<lifeless> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1689EB3170
<lifeless> is the oops I'm trying to duplicate
<mwhudson> branch listings do a query to find the branches the user can see and then pre-populate the permission cache
<lifeless> 3286439196420launchpad-main-slaveSELECT BugSubscription.bug, BugSubscription.date_created ...
<lifeless> blah
<mwhudson> so the security adapter doesn't do any queries
<lifeless> 328 6439 19 6420 launchpad-main-slaveSELECT BugSubscription.bug, BugSubscription.date_created, BugSubscription.id,
<lifeless> is what I'm trying to make my test trigger
<lifeless> but I'm having trouble making it do it
<lifeless> possibly due to caching :)
<mwhudson> lifeless: also, another trick
<mwhudson> lifeless: do you know about LP_DEBUG_SQL_EXTRA ?
<lifeless> do
<lifeless> what doth that do?
<mwhudson> make run LP_DEBUG_SQL=1
<thumper> gives a metric shit-load of output
<mwhudson> prints all queries as the output
<thumper> extra gives a stacktrace too
<mwhudson> LP_DEBUG_SQL_EXTRA gives tracebacks as well as the sql
<lifeless> ok
<thumper> very very handy
<mwhudson> as thumper says,. it's mounds of output
<thumper> in optimisation work
<mwhudson> but can be handy to figure out where that query is coming from
<thumper> I had 70k lines of output for the branch index page
<lifeless> I should capture that for these performance tests
<lifeless> toss it into kcachegrind
<thumper> lifeless: https://code.edge.launchpad.net/~thumper/launchpad/fix-TestSharingMigrationPerformance/+merge/32828
<thumper> lifeless: if you make the mark on that, I'll land the fix
<thumper> lifeless: and that was pretty much the only thing blocking bzr 2.2 landing
<lifeless> mwhudson: so the problem with reproducing the query is that I don't know how to do it *at all*, not just in-a-test, other than 'visit this page that oopses' :)
<mwhudson> lifeless: challenging
<lifeless> even sprinking store.invalidate()'s doesn't trigger it
<lifeless> ah well, so I'll fix the problem I *have* demonstrated.
<lifeless> it would be nice to be able to say
<lifeless> 'now make run should give me this test environment'
<lifeless> or something
<mwhudson> barry had some terrible hacks for that iirc
<lifeless> sync_to_demo()
<lifeless>  ?
<lifeless> do windmill tests hate everyone, or just me ?
<StevenK> lifeless: It's everyone, and life itself
<lifeless> does this mean anything to anyone ?
<lifeless> ======================================================================
<lifeless> ERROR: lp.registry.windmill.tests.test_person_picker.TesPersonPickerWidget.test_person_picker_widget
<lifeless> ----------------------------------------------------------------------
<lifeless> _StringException: Text attachment: garbage
<lifeless> ------------
<lifeless> [<Thread(Thread-657, started daemon 47266133427984)>]
<lifeless> ------------
<wgrant> I see that most times I run that.
<wgrant> So I just sort of ignore it.
<wgrant> While giving it a look of extreme disapproval.
<lifeless> blocking landing is asad
<wgrant> Oh, it did that in EC2?
<lifeless> yes
<wgrant> :/
<lifeless> yes
<lifeless> ><
<lifeless> my registry /participants fix works here
<lifeless> but there is an extra query on prod
<lifeless> sorry
<lifeless> ec2
<mwhudson> lifeless: for reasons i have (shamefully) never really dug into, i find that when i have a genuine failure in ec2, i get a windmill failure too
<mwhudson> when i fix the genuine failure, the branch lands ok
<mwhudson> maybe this is just luck or maybe something else
<lifeless> sigh
<lifeless> a new SELECT Person.account, Person.creation_comment, Person.creation_rationale, Person.datecreated, Person.defaultmembershipperiod, Person.defaultrenewalperiod, Person.displayname, Person.hide_email_addresses, Person.homepage_content, Person.icon, Person.logo, Person.mailing_list_auto_subscribe_policy, Person.merged, Person.mugshot, Person.name, Person.personal_standing, Person.personal_standing_reason, Person.registrant, Person
<lifeless> mwhudson: interesting
<lifeless> I've sent cachedproperty off on its own with the registry branch fix
<lifeless> while I try to figure out wtf I see an extra query even with devel merged in
<lifeless> maybe something landed in the last few hours
<lifeless> oh wow
<lifeless> we trigger a DB lookup on every person lookup via this code path
<lifeless> shudder
<lifeless> yay storm bugs
<lifeless> object cache coherency, can you imagine it?
<mwhudson> it sounds fairly unlikely :-)
<lifeless> har har har
<lifeless> thats twice in 1 month
<lifeless> https://bugs.edge.launchpad.net/storm/+bug/619017
<lifeless> I'm not trusting the layer much now, as a result
<lifeless> I'd rather storm didn't have an object cache.
<lifeless> it would make it much simpler
<lifeless> and more focused
<lifeless> mwhudson: or perhaps you were being sardonic ?
<mwhudson> lifeless: a touch, yes
<lifeless> ha!
<thumper> poolie: ping
<thumper> mwhudson: know the simplest solution to this? http://pastebin.ubuntu.com/479196/
<mwhudson> thumper: :(
<thumper> mwhudson: could trim the input on _show_lline
<thumper> mwhudson: but we'd still have the |
<mwhudson> thumper: let me try to remember what the test is actually testing
<thumper> mwhudson: most of the TestLoggingUIFactory tests fail
<thumper> with similar errors
<thumper> I'm guessing the base class was updated in 2.2
<mwhudson> thumper: yeah, looks like progress reporting changes in bzr
<mwhudson> thumper: possibly needs a code fix, as that's going to make the code import logs look rather lame i suspect
<thumper> :-|
<thumper> I wonder if the _render_bar was renamed
<mwhudson> i'm just updating my bzr branch
<mwhudson> (i think it's fairly ancient)
 * thumper runs away again
<mwhudson> thumper: got a branch i can run tests in?
<mwhudson> I wonder if overriding _avail_width to always return None in LoggingTextProgressView will help
<lifeless> poolie:
<lifeless> ^
<mwhudson> alternatively def _render_line(self): return self._render_bar() will probably do it
<lifeless> mwhudson: so
<lifeless>   File "/home/robertc/launchpad/lp-branches/working/lib/canonical/launchpad/webapp/authorization.py", line 207, in checkPermission
<lifeless>     principal.account)
<lifeless> is that the code path you're talking about w.r.t. checking permissions doing db access ?
<mwhudson> probably
<mwhudson> oh
<mwhudson> is the next call down in canonical.launchpad.security?
<mwhudson> lifeless: ^
<lifeless> checkAccountAuthenticated
<lifeless> yes
<mwhudson> then yes
<lifeless> so the *menu* is generating this
<lifeless> :/
<mwhudson> enabled_with_permission ?
<mwhudson> what's the context object being checked?
<lifeless> dunno
<lifeless> but probably milestone/product
<mwhudson> mm
<mwhudson> lifeless: can you pastebin the whole traceback?
<lifeless> sure
<lifeless> http://pastebin.com/EkXrtUK6
<thumper> mwhudson: yeah, I do
<mwhudson> lifeless: wee, it's checking if the user can review the page
<mwhudson> er
<mwhudson> lifeless: wee, it's checking if the user can review the product
<thumper> mwhudson: lp:~thumper/launchpad/new-bzr
<mwhudson> ReviewByRegistryExpertsOrAdmins
<lifeless> mwhudson: how did you figure that out ?
<thumper> although I'm about to run out yet again
<mwhudson> lifeless: #
<mwhudson>   File "/home/robertc/launchpad/lp-branches/working/lib/canonical/launchpad/security.py", line 232, in checkAuthenticated
<mwhudson> #
<mwhudson>     return user.in_admin or user.in_registry_experts
<mwhudson> -> look at line 232 of security.py
<mwhudson> hi tech haxorring!
<lifeless> oh
<lifeless> I figured that would be generic. serves me right
<mwhudson> it's crummy, but it's a once per page check
<thumper> mwhudson: found it
<mwhudson> thumper: oh?
<thumper> mwhudson: we set _render_bar to return ''
<thumper> mwhudson: the base _render_line method does bar_string = self._render_bar() first
<thumper> then
<thumper> if (task_part or trans) and not bar_string:
<thumper>             bar_string = '| '
<thumper> further down
<lifeless> mwhudson: then why do I see two of them ? :)
<mwhudson> aah
<thumper> :(
<mwhudson> lifeless: both sides of the or execute a query i guess
 * lifeless facepalms
<mwhudson> lifeless: i guess i should have said "fixed number of queries per page"
<mwhudson> lifeless: ORMs r grate
<mwhudson> thumper: i guess overriding _render_line() is one fix
<thumper> mwhudson: yeah
<thumper> I'll look at it again when I get back
<lifeless> mwhudson: want to have a rotation in 6 months or so, to work on a separated mapper approach ? :)
<mwhudson> lifeless: heh
<mwhudson> lifeless: we should rewrite launchpad in haskell and use meta-programming to compute the set of needed queries before we actually start executing code for the page!!
<lifeless> mwhudson: funny you should say that
<lifeless> we do need something along those lines (the needed queries, not haskell)
<mwhudson> lifeless: given the pypy background, perhaps not so much
<lifeless> mwhudson: :)
<mwhudson> it some sense, it would be fairly easy to do the page rendering twice, once to collect the queries and then again for real
<mwhudson> you could even do the first bit ahead of time
<mwhudson> the fun part is conditionals, of course
<ajmitch> fsvo fun?
<mwhudson> you _could_ abstractly interpret the code repeatedly, taking a different path and giving a different answer for each branch
<mwhudson> pypy does exactly this
<lifeless> see
<lifeless> while this sounds extremely promising
<lifeless> I'm actually ok with just putting a cap on the queries
<mwhudson> :)
<lifeless> and failing tests if the cap is exceeded
<poolie> hi thumper, mwhudson
<poolie> mwhudson: all ok now?
<lifeless> how can I tell if an Interface is exported via APIS ?
<james_w> if it has export* decorators on it
<lifeless> ok
<lifeless> so BugTaskSearchParams isn't exported
<lifeless> \o/ I can mess with it
<mwhudson> poolie: i think so, although it's really thumper who's worrying about this
<lifeless> can anyone explain what a 'non conjoined task' is ?
<wgrant> lifeless: A task that isn't conjoined. Do you know what a conjoined task is?
<lifeless> no
<StevenK> lifeless: When you're done with wgrant, do you have time to mumble?
<lifeless> sure
<lifeless> I think I've *finally* found where this private check is kicking in
<lifeless> StevenK: but skype please
<lifeless> it works
<wgrant> lifeless: If you have a task for a project and its development series, the project task is hidden, and becomes a conjoined slave to the development series task.
<wgrant> (that's then you see "Status tracked in SomeSeries")
<lifeless> ok
<lifeless> StevenK: rbtcollins
<wgrant> Setting the status on the series task sets it on the project task too.
 * StevenK grumbles how lifeless hates freedom, and hides
<lifeless> StevenK: do you have skype?
<StevenK> lifeless: Yes
<lifeless> and do you want to talk
<StevenK> lifeless: Yes -- and I cope with mumble's lag every stand-up, so ... :-)
<StevenK> lifeless: So I'm only teasing
<lifeless> arch_tag in [%s]
<lifeless> where the %s is
<lifeless> sqlvalues(tags)
<lifeless> or something
<lifeless> you need ,
<lifeless> 's
<lifeless> so I guess ",".join(sqlvalues(*arch_tags))
<StevenK>             include = "architecturetag IN (%s)" % sqlvalues(
<StevenK>                 ','.join(self.arches))
<lifeless> only
<lifeless> include = "AND whatyou haveabove
<wgrant> sqlvalues(list(self.arches))?
<lifeless> wgrant: I guess
<lifeless> wgrant: but why the list ?
<StevenK> wgrant: But I can't just blat that into a string with %s?
<poolie> lifeless: so i am thinking of just using a textarea for editing the rules
<lifeless> its a tuple already
<poolie> at least to start with
<lifeless> poolie: +1
<poolie> since it's a very nerd-oriented thing
<wgrant> lifeless: Ah, not already a resultset?
<wgrant> s/already/just/
<StevenK> wgrant: Damn it, it's my code. :-) It's a tuple
<thumper> poolie: hi
<wgrant> Just sqlvalues(self.arches) should work, then.
<lifeless> wgrant: does that return a iterable of strings ?
<poolie> hi thumper
<thumper> poolie: I was discussing the issue we have with our code import bzr logger
<thumper> poolie: and that things changed
<thumper> poolie: http://pastebin.ubuntu.com/479196/
<thumper> poolie: is an example of the failure
<thumper> poolie: just working out what to change in our logger
<poolie> huh
<thumper> was thinking overriding _render_line
<poolie> i would guess this is actually a uifactory problem
<poolie> and that these tests want to test TestTextUI
<poolie> but they don't explicitly set it, they just assume it's the default
<poolie> which it may not be in your context
<wgrant> >>> 'foo IN %s' % sqlvalues([1, 2, 3])
<wgrant> 'foo IN (1, 2, 3)'
<wgrant> lifeless, StevenK: ^^
<thumper> poolie: I'm actually guessing part of the issue is line 392 of bzrlib.ui.text.py
<thumper> poolie: as our logger has _render_bar defined to return ''
<thumper> poolie: FYI, this is the lp.codehosting.codeimport.uifactory.LoggingTextProgressView
<poolie> oh this is your test, not mine
<thumper> poolie: yes
<StevenK> LINE 5:             FROM DistroArchSeries WHERE distroseries = 3 ''
<poolie> thumper: i guess what's broken it is that _render_line now does the whitespace filling, not repaint
<poolie> or something like that
<thumper> actually, I'm thinking line 391 should be:  if (task_part and trans) and not bar_string:
<thumper> rather than  if (task_part or trans) and not bar_string:
<thumper> you only need the bar if there are both task_part and trans to separate them
<thumper> perhaps?
<thumper> oh, and the whitespace filling
<poolie> thumper: you seem to have two problems, one being the whitespace padding and the other is the bar
<thumper> yep
<thumper> poolie: I'm thinking it is easier just to override _render_line
<poolie> i agree about the 'and' in that line
<thumper> ok
<thumper> I could just change the _avail_width to return None
<thumper> but we'd still have the bar issue
<poolie> but if you want complete control you should probably just override _render_line
<poolie> to me that is most consistent with the intention
 * thumper nods
<thumper> ack
<lifeless> storm/database.py +236
<thumper> lifeless: whatcha doin?
<lifeless> talking about doing INSERT INTO in storm
<lifeless> compile(expr, State()
<lifeless> )
<lifeless> -> sql
<lifeless> expr - Select(table, And(x, y))
<lifeless> storm.expr.compile_insert
<lifeless> StevenK: so I'm debugging https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1689EB3170
<lifeless> StevenK: do you have lpadmin account access in the dogfood environment ?
<lifeless> (I need to be put in a group there, to trigger this)
<lifeless> if you don't, don't worry, I'll get mthaddon later on staging.
<StevenK> lifeless: Oh, you need a duck on dogfood?
<lifeless> I need to be in the landscape team
<lifeless> I am on prod, now
<lifeless> how that is arranged may need a duck
<wgrant> Um, dogfood is really not a good place to test timeouts.
<lifeless> wgrant: I don't want to test the timeout
<StevenK> lifeless: And yes, I can ... force you into landscape
<lifeless> wgrant: I want to get a definitive backtrace of the cause of the query
<lifeless> StevenK: hold off for a bit
<lifeless> like I said just before our call, I think I'm onto the cause.
<lifeless> bug.py 1561 looks like a clear culprit
<lifeless> though I still need to figure out WTF I can't trigger this in a test
<lifeless> possibly a non-proxied bug object ? surely not.
<lifeless> I'm using userBrowser
<poolie> lifeless: you should try 'ssh launchpad-vm ./bin/test --subunit|tribunal-subunit -'
<lifeless> poolie: :)
<lifeless> ok, so userBrowser goes through publication
<lifeless> wtf
<lifeless> wtf
<lifeless> wtf
<wgrant> Why wouldn't it?
<lifeless> its good that it does
<lifeless> wtf am I not seeing this security code popping up I mean
<lifeless> \o/ cachedproperty branch passed ec2
<poolie> can someone point me to a good example of non-doctest pagetesting?
<lifeless> I pasted an ok example in last weeks perf tuesday thread
<poolie> ah test_archive_packages
<poolie> ok
<lifeless> they seem to generally live in browser/tests/*
<poolie> yep i found some more
<poolie> obvious really
<poolie> the ones that do exist seems pretty clean
<lifeless> man, I hate debugging by printf
<poolie> has anyone ever tried running pgsql on a tmpfs, or with libeatmydata?
<lifeless> yes
<lifeless> not a huge difference
<mwhudson> it helps if you have lots of ram and a slow hd
<poolie> where was the build pydoctor stuff?
<mwhudson> poolie: http://people.canonical.com/~mwh/canonicalapi/
<mwhudson> but there's a good chance apidoc.launchpad.dev is more useful these days
<wgrant> StevenK: Shouldn't your packageset thing be copying memberships too?
<wgrant> At the moment it's just copying empty sets.
<StevenK> Empty sets?
<StevenK> wgrant: I'm about to head out for an hour or so, can you dump what you think I'm missing and how the tests could be extended in a PM?
<wgrant> StevenK: I suspect that the new package sets will have nothing in them.
<StevenK> That can be tested trivially
<wgrant> It can be.
<wgrant> It needs to copy packageset<->packageset and packageset<->package relationships.
<lifeless> \o/ reproduced the query in the test
<lifeless> that makes me so much happier
<lifeless> I thought I was going nuts
<poolie> yay, a pagetest passes
<lifeless> \o/
<poolie> ok, might take a berak
<poolie> tests for merge of the existing flags code into devel are still running
<lifeless> time for a break for me too, that was a marathon head beating session to find out that userBrowser 'works' with the wrong password
<poolie> oh, nice
<poolie> btw if i want to have /+features
<poolie> is it reasonable to say that's a view of ILaunchpadRoot?
<poolie> that doesn't seem quite right but it does seem stadnard
<lifeless> yes
<StevenK> wgrant: Still here?
<wgrant> StevenK: Indeed.
<StevenK> wgrant: Just looking at the branch now
<lifeless> wgrant: btw, registry branch has landed
<lifeless> wgrant: your worst gears may come true next time edge updates
<StevenK> Worst gears?
<StevenK> First, second, third, fourth or fifth?
<StevenK> Damn it, wgrant is right
<lifeless> aboot?
<StevenK> Alpha?
<wgrant> Yay.
<lifeless> StevenK: about
<StevenK> a = []
<StevenK> b = [u'pmount']
<StevenK> wgrant: ^
<lifeless> what causes a security proxy to be wrapped around something
<lifeless> securedutility... ah layers
<lifeless> ...nope
<lifeless> databasefunctionallayer should have sec proxies around utilities, no?
<wgrant> It should, yes.
<lifeless> is there an idiom for checking that?
<lifeless> and it does
<lifeless> hmm
<adeuring> good morning
<StevenK> wgrant: So, which table includes the package names for packagesets?
<lifeless> guess how many queries this takes:
<lifeless> (in the assert call)
<lifeless> target = self.makeBugTarget()
<lifeless> person = self.login()
<lifeless> self.factory.makeBug(product=target, private=True, owner=person)
<lifeless> self.factory.makeBug(product=target, private=True, owner=person)
<lifeless> login_person(person)
<lifeless> tasks =target.searchTasks(BugTaskSearchParams(person, omit_dupes=True,
<lifeless>             orderby=['status', '-importance', 'id']))
<lifeless> self.assertStatementCount(0, lambda:[task.getConjoinedMaster for task
<lifeless>             in tasks])
<lifeless> no takers?
<StevenK> lifeless: 100?
<lifeless> 5
<lifeless> each security check is 2 queries
<lifeless> sanity check: if we'v edone a search limited by a users view, saving that user in the results as someone we know can see them is sane?
<mrevell> Yo
<bigjools> hey wgrant
<lifeless> bigjools: btw, b-m had a hiccup midday, wgrant has the details
<bigjools> lifeless: I read the bug
<bigjools> lifeless: thankfully it's nothing to do with the recent changes
<lifeless> phew
<lifeless> ok, down to 2 queries
<lifeless> lets see if I can go for 1
<lifeless> \o/
<lifeless> milestone pages should be happier soon :)
<bigjools> woot
<lifeless> it may also help a bunch of other private bug related perf issues
<lifeless> due to cachedproperty.
<lifeless> whats the preferred way to spell 'adapt this'
<lifeless> e.g.
<lifeless> if I have a person_or_personroles
<lifeless> is it
<lifeless> person = IPerson(person_or_personroles) ?
<thumper> yes
<lifeless> hmm, I think I'll add 'id' to IPersonRoles
<lifeless> jml: like you, I can't write code without tests
<lifeless> jml: but its ok, its a good place to be
<lifeless> now I have a weird situation
<lifeless> the second page load with more bugs takes less queries :)
<wgrant> StevenK: Sorry, was travelling.
<wgrant> StevenK: PackageSetInclusion and PackageSetSource, IIRC.
<wgrant> There's also PackageSetGroup, but I don't know if that's used for anything.
<wgrant> bigjools: It isn't?
<wgrant> bigjools: It looks like it's meant to be asynchronous...
<wgrant> And given that the whole Deferred management thing changed last week...
<bigjools> wgrant: nothing changed in what is Deferred
<bigjools> it just interleaves them more
<thumper> \o/
<thumper> bzr 2.2 upgrade in the pipe
<lifeless> \o/
<lifeless>  lp.registry.browser.tests.test_milestone.TestMilestoneIndex.test_more_private_bugs_query_count_is_constant
<lifeless>   Ran 1 tests with 0 failures and 0 errors in 2.864 seconds.
<lifeless> ignoring the pathetic sped
<lifeless> -it works- muahahhaaha
<jml> lifeless, \o/
<lifeless> win 68
<lifeless> so
<lifeless> I'm going to push
<lifeless> propose for merge
<lifeless> and foff to relax
<lifeless> @930pm
<lifeless> oh yay, apt-get update ruined ca-cert
 * bigjools switches conversation to -dev
<bigjools> wgrant: I plan  on converting the rest of the sync stuff to async at some point anyway
<wgrant> Yeah, it needs it.
<jml> good plan :)
<wgrant> Although the next significant step is to remove the p-u invocations.
 * bigjools chuckles at jml
<wgrant> That should just about get it to an acceptable speed.
<bigjools> wgrant: jelmer has a branch that does that, it's on dogfood :)
<wgrant> Does it dump it into a directory with a daemon watching it?
<bigjools> cronjob for now, but yeah
<wgrant> Ah.
<wgrant> How does that interact with the build state? It sits FULLYBUILT without binaries for up to a minute?
<bigjools> something like that yeah, but it's still quicker than what we have right now
<wgrant> Probably.
<bigjools> definitely :)
<bigjools> wgrant: any comments on https://bugs.edge.launchpad.net/soyuz/+bug/619088
<wgrant> bigjools: NMAF does it. I don't know why a-f wouldn't.
<bigjools> it's a-f .... all bets are off
<wgrant> True.
<wgrant> Even Debian's trying to kill it now.
<lifeless> ok, have a good performance tuesday
<lifeless> bigjools: I'll pull the performance tuesday topic-element off tomorrow morning - I'd like it to be up there for a full 24 if thats ok:)
<bigjools> lifeless: yeah, did I remove it last week?
<lifeless> bigjools: yeah :)
<bigjools> lifeless: oops. sorry :)
<lifeless> bigjools: de nada
<poolie> is there an official way to make the right pqm command to land a branch i've already tested locally
<poolie> i guess lp-land?
<bigjools> poolie: I do it by hand
<bigjools> old skool :)
<poolie> echo blah|gpg|mail?
<bigjools> you need the pqm plugin for bzr
<bigjools> then I do "bzr pqm-submit -m ...."
<noodles775> lp-land should work.
<bigjools> you can use your approach if you can remember the pqm commands I guess
<noodles775> bigjools: you don't need to, lp-land gets them from the MP.
<bigjools> even betterer, I've not used it
<poolie> OAuth needs a "yeah, yeah" button
<noodles775> heh
<jml> I have come up with the perfect solution for this ravenous hunger: Food!
<poolie> you can have a lamb, leek and mint stew
<poolie> if you can get over here in time
<poolie> urk
<poolie>   File "/usr/lib/python2.6/dist-packages/lazr/restfulclient/resource.py", line 308, in __getattr__
<poolie>     % (self.__class__.__name__, attr))
<poolie> AttributeError: 'Entry' object has no attribute 'source_branch'
<poolie> i know this indicates some internal error but i can't remember what
<lifeless> it may mean Entry isn't an IMergeProposal
<poolie> i've seen that happen when you call into lplib again after previously getting an exception
<lifeless> poolie: red dead redemption is kinda fun
<poolie> yes it is
<poolie> i guess a cache is corrupt
<jml> umm
<jml> poolie, actually, it's because you are giving it a branch URL not a mp URL
<jml> I think
<lifeless> thumper: btw
<lifeless> thumper: CodeImportSchedulerApplication:CodeImportSchedulerAPI - https://devpad.canonical.com/~stub/ppr/lpnet/latest-daily-timeout-candidates.html - click on sort-by-total-queries
<lifeless> thumper: thats doing a _lot_ of db work
<lifeless> only three per hit on avg, but boy - a huge number of hits. Or something like that
<lifeless> wgrant: Distribution:+archivemirrors
<lifeless> wgrant: if you're going to look at perf - that looks to be a classic death-by-sql
<lifeless> 94 hits
<lifeless> 69715 queries
<poolie> hm i got a "failed to verify gpg signature"
<poolie> i might not actually be allowed to submit?
<lifeless> hah, thats possible
<lifeless> losa should be able to fix that fr you
<poolie> ok
<wgrant> lifeless: Is that a typo?
<lifeless> wgrant: is what a typo?
<wgrant> lifeless: 69715
<lifeless> no
<wgrant> queries.
<wgrant> That is insane.
<lifeless> yes
<lifeless> 700 per hit
<lifeless> and you wonder why its timing out :)
<poolie> lifeless: i'll ask tomorrow; could you send https://code.edge.launchpad.net/~mbp/launchpad/flags-webapp/+merge/32833 for me now to reduce latency?
<wgrant> Oh.
<lifeless> wgrant: its still about 100 times too many queries
<poolie> i may put up the edit ui tomorrow, depending
<wgrant> lifeless: I wonder how much we can cut the base query count.
<lifeless> poolie: please specify a commit message
<lifeless> poolie: sounds great on the edit ui
<poolie> done
<lifeless> wgrant: I think you could get it to 8
<lifeless> wgrant: 4 for oauth & other auth cruft, 3 for cookies and session
<lifeless> 1 for the content.
<jpds> lifeless/wgrant: Not timing out here; you two should move to LDN.
<lifeless> jpds: whttps://devpad.canonical.com/~stub/ppr/lpnet/latest-daily-timeout-candidates.html
<lifeless> 1% of hits are timing out
<lifeless> 40% are over 10 seconds long
<jpds> Yeah; the freshness stuff needs fixing.
<lifeless> jpds: its doing *an average* of 700 queries per page load :)
<lifeless> jpds: thats very much 'needs fixing' :)
<lifeless> poolie: ok I think i've told it the magic nuts n bolts to make it do stuff
<lifeless> gnight
<lifeless> tag, somebody is it; make things better
<deryck> Morning, all.
<jml> deryck, good morning
<stub> Do people look at the old patch-???.sql files for examples, or can I just trash them when they are no longer needed? Its about 85 files.
<wgrant> I use them as examples sometimes, but only because I'm lazy.
<wgrant> For everything else, there's VCS history.
<stub> Just wondering if people want them left around for laziness, or trashed for performance.
<StevenK> wgrant: I can't see anything in packagesetinclusion, but I'm about to did through .addSource() code
<wgrant> StevenK: I'm pretty sure it's used for packageset inheritance.
<StevenK> I was afraid of that
<StevenK> I'm already having to write one horrible SQL transform, what's two more :-(
<wgrant> StevenK: What about packagesetgroup?
<StevenK> wgrant: Although, the code for .addSource() only touches packagesetsources, so perhaps packagesetinclusion is done by trigger?
<StevenK> wgrant: I don't touch it at all, currently
<wgrant> StevenK: packagesetinclusion is for packageset inheritance. It doesn't touch sources.
<StevenK> Sigh.
<jml> stub, I vote trashing for performance
<wgrant> And packagesetgroup seems to be to group them across series? Not entirely sure.
<StevenK> wgrant: I'm not either
<jtv> danilos, henninge: about setCurrentTranslation, submitSuggestion & approveâ¦  ISTM we'll be calling setCurrentTranslation only from tests and from the new approval method.  (Doing it this way even for imports would probably simplify the conflict checks as well).  If that is right, that would mean we could handle things like karma in submitSuggestion and approveSuggestion, completely outside of setCurrentTranslation.
<jtv> Oh, and from other scripts perhaps where we most likely don't _want_ karma involved.
<jml> hmm.
<jml> why doesn't Mismatch just take some descriptions and details?
<mars> jml, to reduce coupling between Matcher and Mismatch?
<jml> they are pretty tightly coupled anyway
<mars> well, off the top of my head, lets say you had two matchers
<mars> they each create an instance of the same mismatch class
<mars> if you pass in the error message or change the detail handling, then both matchers are affected
<mars> under the current design only changes to the mismatch constructor API will cause callers to change
<jml> oh, sorry, I wasn't being clear.
<jml> I'm not proposing removing describe() or get_details(), just changing the base Mismatch to take optional parameters for description and details so every single Matcher doesn't need to make its own Mismatch subclass.
<jml> (alternatively, make a SimpleMismatch that does that)
<mars> ah, I was thinking of a similar idea, but using a factory function that returns some Mismatch template class instance
<mars> same diff
<mars> so yes, something like that would be nice for convenience
<mars> jml, looking through testtools.matchers - it might be convenient, but there would be a real temptation to eliminate all the mismatch classes, or for developers to not create new ones.  It would be very easy (too easy?) to just pull the mismatch-specific code (often one line) up into the matcher.
<jml> mars, you mean less code that does the same thing and preserves the current API?
<mars> well, it does the same thing, but in a different form (no more well-named Mismatch subclasses)
<mars> not saying that's a bad thing, just pointing out one consequence of the convenience
<jml> mars, is that such a loss?
<jml> right
<jml> well, the patch is up
<gmb> sinzui, I've just filed https://bugs.edge.launchpad.net/malone/+bug/619218 against malone, but I'm wondering if it should be a registry task instead.
<gmb> What do you think?
<sinzui> I think it is bugs since not all projects use bugs
<sinzui> gmb: is this a feature we are considering?
<gmb> sinzui, No, just something that I've seen mentioned twice in as many days.
<gmb> I'll leave it on bugs.
<gmb> Thanks.
<sinzui> EdwinGrubbs, gmb: I landed a fix to count only open bugs on a dsp for https://edge.launchpad.net/ubuntu/maverick/+needs-packaging Do I need to wait for a daily proc to run that updated bug heat on dsps (max_bug_heat bug_heat_count)?
<EdwinGrubbs> sinzui: I believe that it runs whenever a bugtask is added or removed on a sourcepackage, so they won't all get updated at once.
<sinzui> Thanks EdwinGrubbs
<EdwinGrubbs> matsubara: ping
<matsubara> hi EdwinGrubbs
<EdwinGrubbs> matsubara: to make it easier to debug errors where mailman sends an email to launchpad for moderation via xmlrpc, I want to add the full text of the email to the oops. I've looked at how oops-tools parses it, and I want to discuss the various possible solutions.
<barry> /join #ubuntu+1
<barry>  
<matsubara> EdwinGrubbs, sure, here or mumble? (IRC is easier for me as I'm in the middle of something)
<EdwinGrubbs> matsubara: we can do it on irc. Before I start listing off the possibilities, do you have any thoughts on how you would like this to be solved?
<gary_poster> sinzui: hey.  does registry do gpg stuff usually (https://bugs.launchpad.net/bugs/618347) or is this a foundations thing?  foundations hasn't done much of any gpg stuff on my watch.
<sinzui> gary_poster, That is registry. I believe it is the last piece of code that uses logintoken
<jcsackett> lifeless, ping?
<gary_poster> ok thank you sinzui
<EdwinGrubbs> matsubara: ok, I'll start then. Option 1) Just add a delimiter at the end of the traceback to add a single section.
<EdwinGrubbs> matsubara: Option 2) Use the python multifile module to add mulltiple delimited sections after the traceback.
<matsubara> EdwinGrubbs, sorry, I'm looking for an oops as an example. I think one idea is to make like the codehosting oopses and add the email as part of the request variables
<matsubara> kinda hackish though
<matsubara> EdwinGrubbs, option 1 is interesting
<EdwinGrubbs> matsubara: but aren't the request variables limited to one line normally. Even if the oops-tools can parse a single req var that is multiple lines, it would probably look really bad when browsing oops files on devpad.
<matsubara> I just talked to the ISD guys and they want to have a way to parse oopses like this: https://admin.isd.canonical.com/oops/?oopsid=1607carambola1 and render the extra data info as html. perhaps extra_data should be a section after the traceback
<jml> james_w, are your html matchers available for use in Launchpad (sorry, I've asked you before)
<james_w> jml: I don't think so
<jml> james_w, where could I find them?
<james_w> lp:soupmatchers
<EdwinGrubbs> matsubara: Option 3) expand on the email formatting idea. Add a header to indicate a new oops format, and turn it into a multipart/mixed mime email. This has the added benefit of providing name/value pairs for each part and unlimited number of sections.
<jml> james_w, thanks.
<matsubara> EdwinGrubbs, option 3 sounds interesting
<EdwinGrubbs> matsubara: my one concern about rendering html in an oops is that the html might contain javascript attacks.
<matsubara> EdwinGrubbs, yep, I'm aware of that. one of the things I'd like to discuss with the ISD guys before they implement that option
<EdwinGrubbs> matsubara: so, should I wait until we can get more info from ISD, or can one of the solutions be started on. Technically, the oops-tool can just format extra data as text now and be updated to not escape the html later when the security concerns are addressed. The mime solution would be the handiest for that, since we would be able to add a content type to each section.
<matsubara> EdwinGrubbs, I think we can start and then adapt the work to address ISD's needs in the future.
<matsubara> EdwinGrubbs, once you get the changes needed on the LP side could you ping me so I can update oops-tools to understand this new format?
<EdwinGrubbs> matsubara: well, I was planning on updating oops-tools at the same time for testing purposes. Do you want me to go with Option 3?
<EdwinGrubbs> matsubara: the main drawback to option 3 is that oops-tools has to be updated before launchpad starts producing oopses in that format.
<EdwinGrubbs> just appending to the end of the traceback is sorta backwards compatible.
<matsubara> EdwinGrubbs, right. I'd rather go with option 3 given that we will need this work in the future anyway
<EdwinGrubbs> ok
<jcsackett> what time does lifeless usually get active on IRC?
<leonardr> i have a question for anyone who might know
<leonardr> i have a test that's suddenly started failing on ec2
<leonardr> and the failures would be explained if the AppServerLayer used to make Launchpad available at "launchpad.dev", but recently it was changed to be available at "launchpad.dev:8085"
<leonardr> did anything like that happen, or has the test launchpad always been available at launchpad.dev:8085?
<leonardr> i know for some test layers it's been at :8085, but maybe it was formerly inconsistent and the inconsistency was fixed recently?
<leonardr> another possibility is that the tests used to be run with a sitename of 'testrunner' and now they're run as 'development'
<jml> jcsackett|afk, early NZ time.
<jml> leonardr, sorry, I don't know of any recent changes along those lines.
<jml> leonardr, I'd be very surprised if the latter possibility were true.
<leonardr> well, damn
<jml> leonardr, does the test fail locally?
<leonardr> jml: no
<leonardr> jml, more specifics about the test
<jml> leonardr, even if you merge in the latest devel? it's possible that something broken landed recently.
<leonardr> jml, i think i know why it doesn't fail locally, let me explain
<jml> leonardr, please do.
<leonardr> when you run 'make' in launchpad, it generates some wadl for your instance
<leonardr> the wadl defines the root url of the wadl as being http://api.launchpad.dev/
<leonardr> s/of the wadl/of the web service/
<leonardr> in local usage (and i'll test this to make sure), when you run a test, launchpad starts up as launchpad.dev, and the launchpadlib test makes requests to http://api.launchpad.dev/, and gets wadl that tells it to make more requests to http://api.launchpad.dev/...
<leonardr> on EC2, when you run a test, launchpad starts up as launchpad.dev:8085
<leonardr> the launchpadlib test makes a request to https://api.launchpad.dev:8085/
<leonardr> and gets wadl that tells it to make requests to https://api.launchpad.dev/
<leonardr> but there's no such host, and the test fails
<jml> I see.
<jml> I'd be disappointed if ec2 tests were running with a different configuration to the local test runner.
<leonardr> gary, jml: i have 100% confirmed that ec2 environment differs _somehow_ from up-to-date local environment
<leonardr> such that the WADL served by ec2 sends you to api.launchpad.dev, but the WADL served by a local instance keeps you on api.launchpad.dev:8085
<jml> leonardr, I guess moving from fear to unpleasant certainty is an incremental improvement.
<leonardr> i'm also fairly certain that the cached WADL is not contributing to the problem, because the problem persists even when i move the cached wadl out of the way on ec2
<jml> leonardr, do you know if this difference exists on up-to-date stable?
<leonardr> jml: no, i don't
<leonardr> ideally i would be able to watch the wadl being generated on ec2, but i can't, because the problem only shows up during the launchpadlib tests. and a breakpoint in launchpad, triggered by the launchpadlib tests, will just hang, since launchpad is running in the background
<jml> yeah. interprocess bugs suck.
<leonardr> testing another hypothesis...
<leonardr> ok, even disabling the wadl cache altogether on ec2 doesn't get rid of the problem. so there is one specific request that goes to api.launchpad.dev:8085, and launchpad thinks "ok, what is my hostname? aha! it's api.launchpad.dev!" and generates the wadl accordingly
<jml> mars, you might be interested in reviewing: https://code.edge.launchpad.net/~jml/launchpad/different-ec2-mail/+merge/32905
<jml> mars, it's the first deliverable chunk of work from the branch we started at the epic
<Green00000> hi
<Green00000> i have a problem with deleting my launchpad-account ...... can anybody answer??
<benji> Green00000: what's the problem?
<Green00000> hi.
<Green00000> i loged in first.
<Green00000> then i deactivated the account.
<Green00000> wait.
<Green00000> okay.
<Green00000> once again.
<Green00000> .
<Green00000> .
<Green00000> .
<Green00000> i log in launchpad.
<Green00000> then the point "change details"
<Green00000> down on the site
<Green00000> "Deactivate your account"
<jml> Green00000, https://answers.edge.launchpad.net/launchpad/+faq/968 might be helpful.
<Green00000> the message "Your account has been deactivated."
<Green00000> but it's just possible to login.
<Green00000> okay.
<Green00000> read.
<Green00000> but
<Green00000> ....
<Green00000> why can i log in any more??????
<jml> Green00000, you mean, you can still log in even after deactivating?
<Green00000> (btw -- thx to jml.)
<Green00000> yes.
<jml> Green00000, I don't know why that's the case. It probably has something to do with Launchpad using the Ubuntu One account service.
<Green00000> i loged out and delete all cookies.
<Green00000> but the log in with my emailadress and the password is still working.
<leonardr> gary, jml: it's official. i get exactly the same error when i run launchpadlib on an unaltered 'stable' on ec2
<gary_poster> huh
<gary_poster> well, my ec2 instance is *almost* ready to go...
<jcsackett|afk> mars, ping
<leonardr> so the other question is, why is it possible for anyone to land anything?
<mars> hi jcsackett
<jcsackett> mars: you're oncall reviewer today, correct?
<gary_poster> yeah, that question occurred to me
<mars> jcsackett, why yes, I am :)
 * mars does not like that question
<jcsackett> mars: awesome! i've got two mps that sinzui may be getting to, but indicated he didn't mind getting beaten to the punch, and i just realized the mp notices went out long enough go they may have been lost. https://code.edge.launchpad.net/~jcsackett/+activereviews
<mars> (about the ec2 landings)
<jcsackett> mars: don't hate me. :-)
<mars> jcsackett, my bad, crossed messages
<jcsackett> mars: cool.
<sinzui> jcsackett, I started https://code.edge.launchpad.net/~jcsackett/launchpad/unsubscription/+merge/32680 a few minutes ago
<sinzui> you will hate my nitpicks
<jcsackett> sinzui: cool.
<jcsackett> so, mars: only one mp. :-)
<mars> jcsackett, sure, I can review those.  Head on over to #launchpad-reviews and add what you have to the queue
<jcsackett> sinzui: i never hate your nitpicks. :-)
<mars> jcsackett, depends if I or sinzui gets to them first
<leonardr> gary: i have a hypothesis why other people have been landing things. if you run the full test suite it hangs, and after about 6 hours the ec2 instance silently shuts down. if someone is using the --merge option (or whatever it is, i don't use it), maybe the ec2 script interprets this silence as success?
<gary_poster> I hope not leonardr.  mars, any evidence to the contrary? ^^^
<leonardr> and we don't have the problem in production because production servers don't have the :8085 trickery
<mars> ergh - lost with that.  leonardr, your branch fails on stable ec2?  Or just plain old stable ec2?
<leonardr> mars: plain old stable ec2
<leonardr> there are test failures and a hang when running the launchpadlib tests
<mars> that is weird
<mars> leonardr, how about when you run just the one test?
<leonardr> mars: same thing
<mars> ec2 test -o '-t introduction.txt'
<leonardr> oh, start up a new ec2 instance just to run the one test?
<leonardr> i can try that
<mars> leonardr, it is introduction.txt that it hangs on so far?
<leonardr> mars: yes, there are test failures in other launchpadlib tests, but introduction.txt causes the hang
<mars> leonardr, two tests then: first, just introduction.txt.   Next, the suite that introduction.txt is contained in.
<leonardr> ok
<mars> leonardr, don't bother with the second if introduction.txt alone hangs
<leonardr> mars: i'll test with stable?
<mars> leonardr, you don't have to
<mars> stable and devel don't drift that far
<leonardr> ok, but i'll test with the basic code instead of the code with my changes in it
<mars> and there is only moderate risk of you stepping on someone else's accident while trying this
<mars> right
<mars> leonardr, run with --attached if you want to get a better idea when the test finishes
<mars> instead of busy-waiting on your INBOX for the failure mail
<leonardr> ok
<leonardr> i predict there won't be a failure mail--i never got one before
<mars> well, if you run just that one test, it should be pretty obvious when it is hung :)
<mars> ... kind of.  (I hope output buffering doesn't bite us here)
<gary_poster> leonardr: ok, I think I have duped.  I ran ./bin/test -vvt launchpadlib and launchpadlib/docs/introduction.txt has been sitting for about a minute.
<leonardr> gary, did you get any other failures?
<gary_poster> no leonardr
<leonardr> gary: control-c and paste me the traceback
<leonardr> i kind of suspect there are 2 problems
<gary_poster> leonardr: http://pastebin.ubuntu.com/479533/
<gary_poster> that doesn't look very wadllib-y
<leonardr> gary: run bin/test -vvt hosted-files.txt and see if that works
<gary_poster> k
<salgado> rockstar, I've used lp-submit and the m-p was created but I was redirected to the wrong url (https://code.edge.launchpad.net/1.0/~salgado/launchpad/upload-policy-utility/+merge/32911).  it has a leading '1.0'
<salgado> is that a known bug?
<rockstar> salgado, probably not a known bug, but I see why it would do that.
<gary_poster> leonardr: it passed
<gary_poster> this is with your branches, btw, leonardr
<leonardr> gary: yes! that's the problem i originally had on thursday (and, like you, i didn't have the wadl problem)
<salgado> rockstar, should I report it against bzr or is this a plugin?
<leonardr> and i think that is because of the new launchpadlib--the test launchpad thinks it's launching a web browser
<gary_poster> so I should try to dig into that bug, leonardr?
<leonardr> but there could be any number of causes, and it could even be caused by an inaccurate wadl file
<leonardr> gary: yes, please. put a breakpoint in the TestableLaunchpad constructor. coming up with more specific advice now
<gary_poster> ok
<jam> hey all, what is the correct way to run a subset of the lp test suite? Is it just 'bin/test -m lp.module' ?
<gary_poster> jam, use -t
<gary_poster> (my suggestion)
<jam> gary_poster: in what way?
<jam> (can you give an actual invocation)
<dobey> how does one set the status on a bug_task via launchpadlib? task.setStatus() doesn't seem to exist, and task.status = status gives me a HTTP 401 :(
<gary_poster> ./bin/test -t launcpadlib
<gary_poster> jam, I suggest searching for -t TEST in the ./bin/test --help output
<leonardr> dobey: how did you set up your Launchpad object?
<rockstar> salgado, bzr-pipes has an implementation of it.
<jam> gary_poster: you guys really need to play with the bzr selftest suite... this is rather, uh, clunky. :)
<rockstar> salgado, it might be a bzr bug, but let abentley sort that out.
<gary_poster> jam, k :-)
<dobey> leonardr: launchpad.Launchpad(creds, EDGE_SERVICE_ROOT, cachedir)
<jam> so far, I'm sitting at a couple of minutes, and I don't think I've actually seen tests being run yet
<leonardr> dobey: how did you get the creds? if they don't have write permission, then attempting to change the dataset will give you 401
<jam> time bzr selftest -s bt.test__chk ... ran 50 tests in 1.4s
<leonardr> gary: ok, the section 'authorizing the request token' in launchpadlib.txt deserves to fail. at the very least, it prints out stuff like 'come back here and press enter', which isn't there anymore
<leonardr> gary: but i bet it's hanging there instead of printing stuff out
<jam> also not sure why there are 3 200MB instances of python running
<mars> jam, three instances when running bin/test?
<jam> mars: yes
<mars> might be the servers from different layers
<dobey> leonardr: is task.status = status the correct way to do it?
<leonardr> dobey: yes
<leonardr> task.status = status; task.lp_save()
<gary_poster> that's what I was thinking, mars, but not sure this is particularly productive.  old ground.
<jam> mars: one is the librarian it seems
<salgado> rockstar, are you suggesting that I file it against bzr-pipes?  that seems weird -- the help for that command says "From:     plugin "launchpad""
<mars> gary_poster, yep
<jam> mars: two of them are 'python bin/test'
<dobey> leonardr: ok, thanks
<mars> jam, ah, that is because the zope testrunner fires up subprocesses of itself while testing
<rockstar> salgado, that means you should file the bug on bzr then.
<jam> mars: it also seems to be doing stuff like 'branch-distro.py'
<rockstar> salgado, there are two implementations, one in bzr and one in bzr-pipes
<jam> is this all setup stuff?
<jam> Does it run on every test suite run?
<mars> jam, what test command did you run?
<jam> mars: bin/test -t lp.codehosting
<gary_poster> leonardr: going to launchpad-foundations
<leonardr> ok
<jam> I'd *like* to just run the 1 test I just wrote
<jam> but I'm still sorting out how to ever run the test suite
<jam> I'm also seeing it say:
<jam> https://lists.ubuntu.com/archives/bazaar/2010q3/069549.html
<mars> jam, what was the file name of the test you just wrote?
<jam> sorry bad paste
<jam> lib/lp/codehosting/tests/test_lpserve.py
<jam> I also see my invocation saying it is running the 'canonical.testing.layers.BaseLayer' tests
<jam> which seems like they should be skipped by '-t'
<lifeless> moin
<salgado> rockstar, I think in my case it's the one in bzr because I don't seem to have bzr-pipes installed. thanks for the help
<jam> did I type something wrong?
<jam> hi lifeless
<jam> lifeless: thank you very much for making 'bzr selftest' what it is today :)
<lifeless> jam: my pleasure
<rockstar> salgado, no prob.
<mars> jam, 'bin/test -t test_lpserve'  or just 'bin/test -t lpserve'
<jam> mars: any idea how I can stop it gracefully?
<mars> ^C
<mars> and wait
<jam> seems to need waiting for a *long* time
<mars> yep
<jam> it did eventually stop, though
<salgado> I don't wait; I press ^C multiple times
<salgado> but it's either that or wait
<jam> any ideas why 'lp.codehosting' is running canonical.* tests?
<jam> I guess 19s to run the one test file isn't too bad
<jam> though 19s real time to 'Ran 4 tests ... in 7.3s' is a bit high
<jam> (the problem with -t is I'm guessing it has to load all possible tests, and then filter them, which is why bzr switched to '-s' for prefix matching, and only loading tests that could possibly match the pattern)
<jam> anyway, thanks mars and gary_poster I at least have it running
<mars> jam, np.  You are right about the -t thing, discovery takes a while.  IIRC discovery parses every doctest it finds, too
<mars> not optimal
<mars> I looked into fixing that, found the code, but no solution jumped out at me.
<jam> mars: so bzr went the route of using a custom TestLoader and modules that expose 'load_tests' (which is also added to python 2.7 and unittest2)
<jam> so it loads the root, which knows about loading children, and all check against the pattern to see if any of these children should be evaluated
<lifeless> sinzui: jc sackett pinged me, do you know what about ?
<sinzui> I think it was about the review of his branch
<mars> jam, makes sense, some adaptation of that may work for our pagetests too.
<mars> However, I don't know if our testrunner makes a distinction between load versus find
<mars> I know that for our pagetest suites our code does not make that distinction
<jcsackett> lifeless: i just wanted to make sure i had answered your questions re: plus-participation.
<lifeless> jcsackett: ok, let me look up the response now
<lifeless> jcsackett: I tried to tab complete your name here, before but failed - sorry! (Thats why I asked sinzui :P)
<jcsackett> lifeless: it's all good.
<lifeless> ok
<lifeless> so, I've put forward a possible algorithm to do this in two queries
<lifeless> if you liked the previous UI
* lifeless changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 1 of 10.09 | PQM is OPEN | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews
<jcsackett> lifeless, was that to me?
<lifeless> jcsackett: yes
<lifeless> jcsackett: just now, in the MP
<lifeless> poolie: webapp flags is in devel now
<lifeless> gmb: ^
<jcsackett> lifeless: i think i like that, but the graph part goes by a little quickly (or may in fact go over my head).
<lifeless> oh, I missed a step, darn
<jcsackett> sinzui: i've pushed up the changes on my subscription mp.
<sinzui> thanks
<jcsackett> lifeless: that makes me feel a bit better.
<lifeless> updating in a sec
<lifeless> updated
<lifeless> flacoste: thank you
<flacoste> bac, sinzui: so official_codehosting where EXISTS (SELECT 1 FROM Branch JOIN ProductSeries ON (Branch.id = ProductSeries.branch) JOIN Product ON (ProductSeries.id = Product.development_focus) WHERE BranchType = HOSTED AND ProductId = <product_id>)?
<flacoste> i mean it becomes
<flacoste> or can we consider a MIRRORED or REMOTE branch has officially codehosting?
<sinzui> flacoste, HOSTED only
<flacoste> sinzui: ack
<lifeless> jcsackett: does it make more sense now ?
<jcsackett> lifeless: i think so; graph theory isn't my forte, so i'm sort of mulling it over in a terminal to get a handle on it.
<lifeless> ok
<lifeless> please do ping me
<lifeless> if you want to chat about it
<jcsackett> lifeless: will do. probably shortly. :-P
<flacoste> sinzui: that actually gives us about ~2000 more official_codehosting projects :-)
<flacoste> sinzui: make that 4000!
<flacoste> sinzui: about the same number than we have projects officially using Launchpad for Bugs
<sinzui> yes. as back suspected when he learned we were counting it. He was pretty sure that users could not set the value over the last year
<sinzui> s/as back/as bac/
<flacoste> sinzui: looking at https://lpstats.canonical.com/graphs/ProductsFlaggedOfficial/, I'd say they can't flag it since May
<sinzui> I would have though March, but May is close enough
<flacoste> thumper will be happy
<sinzui> flacoste, lets add a constraint product.active = TRUE
<flacoste> sinzui: ok
<flacoste> sinzui: 2000 less projects
<sinzui> We had a lot of project try launchpad.
<jcsackett> lifeless: you want me to PM you about this or just carry on the conversation for all to see?
<lifeless> jcsackett: more eyes, more thoughts
<sinzui> I saw a few hundred projects with empty branches in my review of all projects. There never was code for a lot of projects
<jcsackett> lifeless, works for me.
<jcsackett> okay, lifeless, if i understand you right, you suggest we pull all teams at once, then pull all the relationships for those teams, and out of the resulting query sets map out the relationships.
<jcsackett> so if we pull user in a, b, and c, and then pull that a is in b and c, and then that b is in c, we can build out the link from that.
<lifeless> right
<lifeless> user in a,b,c is one query
<lifeless> a in b,c    and b in c    is one query
<lifeless> but - the second query would be direct memberships only so it would look like
<lifeless> user in a,b,c
<lifeless> user in c           c in b              b in a
<sinzui> flacoste, I think the ProductsFlaggedOfficial graph is suspicious. Blueprints is not used as much as it is claimed to be used
<lifeless> as the two resultsets
<jcsackett> wouldn't a in b,c and b in c be two queries? or are you thinking (in pseudo sql) something like "select team_participant, team, where team_participant is in [list_of_teams]"
<lifeless> well it won't be the team participation exploded table
<lifeless> it will be the direct table
<lifeless> uhm, looking
<lifeless> and yes, 'in' is operator
<lifeless> we want all the rows that directly list any of (user, a, b, c) as the member of a team
<lifeless> memberships = store.find(TeamMembership, TeamMembership.person in (user, a, b, c))
<lifeless> print list(memberships)
<lifeless> [(user, c), (c, b), (b, a)]
<lifeless> # thats fiction, but pretty darn close to reality
<lifeless> we can actually do all of this in one query
<lifeless> I've been describing two to show the algorith,
<lifeless> mwhudson: you were right about the windmill error being bogus
<jcsackett> so, lifeless, basically once we have that final dict you suggest walking through it to find the paths from user to indirect teams? seems like that could get expensive computationally in these pathalogical cases.
<lifeless> walk in the other direction
<lifeless> you have all the terminal points
<lifeless> they all walk directly to the user
<lifeless> bam
<lifeless> the api you have - 'find path to ' is a wonky api
<lifeless> we can just iterate this graph and spit out all in a tabular form all the paths we want to render
<lifeless> then in the template just loop on the paths rendering them
<mwhudson> lifeless: it's very strange that
<lifeless> mwhudson: eparse
<mwhudson> lifeless: the windmill error thing is very strange
<lifeless> yes, yes indeed
<lifeless> jcsackett: if we start at the user, and for every team they are in (direct connections in that graph), we start a path - [user, teamx], and then for each path we repeat that expansion - [[user, teamx], teamy] and so on
<lifeless> jcsackett: that will be fast - we don't permit cycles AIUI in the db
<lifeless> there may be multiple paths, but thats ok. when we're done we just tabulate and discard any duplicate terminal nodes
<lifeless> so we'll have
<jcsackett> lifeless: AIUI == ?
<lifeless> [user, teamx]
<lifeless> As I Understand It
<jcsackett> ah.
<lifeless> [user, teamx]
<lifeless> [user, teamx, teamy]
<lifeless> etc
<lifeless> len(path) !=2 => indirect path
<jcsackett> lifeless: okay, i think i've got it.
<lifeless> oh final tweak I think - prefer the shortest paths when trimming -
<lifeless> [user, teamy] should win over [user, teamx, teamy] in the discard stage
<jcsackett> lifeless: yeah, i sort of assumed that when you mentioned discards.
<jcsackett> lifeless: i do think i like that better; it's worth a shot at least before basically killing the UI.
<lifeless> cool
<lifeless> I think for nearly all of LP perf problems we can do something like this
<lifeless> its been the experience [for most] issues in bzr anyhow
<lifeless> and bzr has a harder problem in many ways : no persistent database with hot disk caches :)
<jcsackett> that does seem like a much harder issue. :-)
<lifeless> losa ping
<lifeless> I want to know what our current ssl session lifetime is set to
<lifeless> and also the SSLSessionCache setting
<mbarnett> lifeless: SSLSessionCache        shmcb:/var/run/apache2/ssl_scache(512000)
<mbarnett> SSLSessionCacheTimeout  300
<lifeless> thanks
<lifeless> rt'd
<mbarnett> welcome
<lifeless> flacoste: rt 40918
<flacoste> lifeless: bumped up
<lifeless> so what makes in_admin etc work
<lifeless> my new branch made a change that looked fine and works some of the time but not always
<lifeless> should one adapt to IPersonRoles?
<thumper> lifeless: do you realise that it takes 30 minutes for a trivial branch to land on devel using PQM?
<thumper> AFAICS much of that time is cleaning the working directory
<lifeless> yes
<thumper> why does it take so long?
<lifeless> would like to fix
<lifeless> thumper: its doing a heavyweight branch
<lifeless> because we have calls that look at the bzr history but need to run in the chroot
<lifeless> if we do a lightweight checkout of the http readonly branch url, they would work
<lifeless> and we could bzr switch to do the commit
<jelmer> 'evening lifeless, thumper
<lifeless> hi jelmer
<mwhudson> gary_poster: hi, can i ask you a couple of questions about buildout?
<lifeless> Segmentation fault in cron mail worries me
<lifeless> on pear
<jelmer> pear.. that's one of the importds isn't it?
<mwhudson> yes
 * rstat1 pokes salgado
<mwhudson> gary_poster: nm, afk
<jam> anyone know where you get "zope.preferences.interfaces.IPreferenceGroup" from ?
<jam> I just did 'apt-get update' and now I can't run the test suite anymore without tracking down stuff like this
<wgrant> jam: Run 'make'
<wgrant> It's an egg that was added a few weeks ago.
<wgrant> You might also need to update-sourcecode, if you're really out of date.
<jam> wgrant: to give a fuller description, I ran apt-get update, which broke bzr-builder because it imported something bzr-builddeb didn't like, when I updated bzr-builder it broke my source code. when I merged devel, it broke because it was missing interfaces
<jam> wgrant: is this par for the course?
<flacoste> thumper: https://lpstats.canonical.com/graphs/BuilddLagProductionSupportedArch/
<jam> (of doing development on launchpad)
<lifeless> jam: sometimes
<lifeless> jam: bzr 2.2 just landed in devel
<lifeless> jam: so you need to do ./utilities/update-sourcecode
<lifeless> jam: and I'd like to fix this up, move to totally deb dependencies, that sort of thing; however this isn't [yet] a widely held view.
<lifeless> jam: I think its stockholm syndrome.
<flacoste> thumper: https://lpstats.canonical.com/graphs/BuilddLagPPASupportedArch/
<jam> lifeless: I think you need 1 dependency structure
<jam> I'm not sure that it has to be debs, since I don't really want apt-get update to break my dev environment each time
<lifeless> jam: I'm not sure why it would break your dev environment
<lifeless> but yes
<jam> (I need the latest firefox patch, and oh, my test suite won't run anymore)
<jam> lifeless: see above. It seems that we use the system bzr-builddeb? but the source tree bzr-builder?
<lifeless> jam: well, given we use windmill, we do have dependencies on firefox
<lifeless> jam: its a little magic, we'll use one if its available or an egg if its not.
<rstat1> s
<rstat1> whoops
<jam> lifeless: If the answer is "don't update your system until your current patch has been landed" I guess that would be a different answer
<jam> I realize it is a significantly more complex project.
<lifeless> jam: I get the impression thats what folk do. IMBW
<jam> but stuff like this is really pretty bad
<lifeless> jam: its got many more dependencies, code base isn't that different in size
<wgrant> I rarely find that an update breaks it.
<wgrant> Except on a dev series.
<wgrant> And even then it's normally the autosync that kills things.
<flacoste> thumper: https://lpstats.canonical.com/graphs/CodeRecipeBaseTypeCounts/20090818/20100818/
<jam> lifeless: as an outsider, it isn't like I can change things. But I do find it strange that it is the current status quo
<jam> as you say, stockholm syndrome :)
<lifeless> jam: right
<lifeless> jam: we're improving the dev process at the moment via the release-features-when-done project
<jam> *sigh* and after all the update-sourcecode, etc. it is still broken
<lifeless> jam: once that is bedded in, other bits of friction will be bubbling to the top
<lifeless> jam: ok, current issue ?
<jam> "DeprecationWarning: please use 'debian' instead of 'debian_bundle'
<lifeless> check in aptitude that you have updated the lp deps
<jam> from bzrlib.plugins.builder import RecipeParser imports something from bzr-builddeb and builddeb complains
<lifeless> apt held-back that update for me for some reason
<lifeless> poolie: are you successfully running concurrent vm tests ?
<poolie> in multiple vms? yes
<lifeless> \o/
<lifeless> you might like to write that up
<poolie> iwbn to automatically CoW-fork them
<poolie> so i can have as many as i like
<lifeless> as well as your please-use-flags mail :)
<poolie> but i haven't got there yet
<lifeless> flags is in devel how
<lifeless> *now*
<flacoste> thumper: https://devpad.canonical.com/~stub/ppr/lpnet/latest-daily-timeout-candidates.html
<poolie> mm i have a draft of that but wanted it to actually be landed first
<poolie> thanks for that
<lifeless> anytime
<jml> I have a big hairy patch to ec2test/remote.py that I would love to have reviewed.
<rstat1> and I have an issue with openid if anyone is interested :)
<wgrant> rstat1: What's the problem?
<lifeless> rstat1: you probably need to describe the issue
<rstat1> https://answers.edge.launchpad.net/launchpad-foundations/+question/120150 << This explains it quite well.
<jml> https://code.edge.launchpad.net/~jml/launchpad/different-ec2-mail/+merge/32905
<flacoste> bac, sinzui: https://code.edge.launchpad.net/~flacoste/tuolumne-lp-configs/fixup-officical_codehosting
<rstat1> the jist of the matter is: authenication on a lp.dev instance i have on a Lucid VM is impossible. I get an oops. "DiscoveryFailure: No usable openid services found for https://testopenid.dev
<wgrant> It sounds like an Apache config issue.
<rstat1> Thats's what I'm thinking...https://testopenid.dev redirects to lp.dev and https://testopenid.dev/+openid coughs up a 503
<rstat1> BUT my config for apache comes straight from the lp repos...save for a modifcation to only use one ip address.
<lifeless> jml: I saw it
<lifeless> jml: I think its great but a pain to review :(
<wgrant> rstat1: What't the 503? Just a normal Apache one?
<bac> flacoste: that branch looks good.  i am not allowed to review it, though.
<wgrant> What does the Apache error log say?
<bac> flacoste: and thanks for doing it!
<rstat1> Yea its a normal 503...I'll go dig up the log.
<flacoste> bac, a losa need to review and deploy it
<bac> flacoste: right.  i didn't know if you pasted just a FYI or wanted a review.
<rstat1> My apache log doesn't appear to have anything useful in it...just a bunch of debug crap from OpenSSL
<jml> lifeless, yeah, I'm sorry about that.
<jml> lifeless, I'm not sure I could stand the pain of breaking it up into multiple patches though.
<lifeless> man, some of these failures are nuts
<rockstar> Hm, sourcepackagerecipebuild views don't really have unit tests, just leaky sourcepackagerecipe view tests.
<lifeless> oh
 * lifeless hates on prejoins
#launchpad-dev 2010-08-18
<lifeless> wow
<lifeless> some hair shows up when you dig
<lifeless> found a code path that disables prejoins for speed
<lifeless> (ok, fine so far)
<lifeless> then, in the inner code it was calling, there is an if block that uses Storm rather than sqlobject joins, and so the prejoins don't get disabled ...
<wgrant> Slow branch scanner makes me sad :(
<lifeless> latency, throughput or capacity?
 * rockstar picks all three
<wgrant> Well, it took nearly four minutes from push to bug link.
<rockstar> wgrant, that's odd.  I've never seen it take more than two.
<thumper> rockstar: call?
<rockstar> thumper, I thought you'd never ask.
 * rockstar repeatedly slaps skype and pulseaudio
<lifeless> oh ffs
<lifeless> sqlobjectresultset is evil
<rstat1> why's that?
<lifeless> because it doesn't implement IResultSet
<wgrant> Isn't that the point?
<lifeless> so stuff like canonical.lib.components.decoratedresultset fail spectacularly when wrapping SQLORS
<lifeless> wgrant: well, we want to migrate
<lifeless> wgrant: yes
<rstat1> you'd think it would be...given the name
<lifeless> rstat1: yes
<rstat1> got anymore ideas as to why my openid doesn't work?
<lifeless> rstat1: I don't, sorry : I haven'ted poked at openid at all yet
<rstat1> ahh...well ok.
<Ursinha> lifeless, hi
<lifeless> Ursinha: hi!
<Ursinha> lifeless, so, are you willing to work on the rollback thing? :)
<lifeless> yes, I'd love to
<lifeless> I'm putting it in my queue right after fixing the fallout from trying to use DRS around a SQLRS
<Ursinha> lifeless, cool
<wgrant> rstat1: What if you revert to the stock LP Apache config?
<lifeless> as its a different branch, I can work on it concurrently
<lifeless> Ursinha: remind me of the log
<Ursinha> lifeless, mars worked today on buildoutifying qa-tagger
<lifeless> qa-rollback means 'it has been rolledout
<Ursinha> I'm merging then you can branch it
<rstat1> wgrant: I could try that..One seccond
<lifeless> and thus qa-rollback means 'can deploy'
<Ursinha> lifeless, yes, qa-rollback is, well, a rollback, so it's safe to land
<Ursinha> I guess it's well described on a wiki page
<Ursinha> let me find the link
<lifeless> Ursinha: so, its a one-liner I think.
<lifeless> one sec
<Ursinha> lifeless, https://dev.launchpad.net/QAProcessContinuousRollouts
<Ursinha> lifeless, tagger also has to identify which revision they're rolling back
<lifeless> lp:~lifeless/qa-tagger/qa-rollback
<lifeless> Ursinha: devs can manually tag that initially though, right? same as tagging qa-ok ?
<rstat1> well of course it'd work when I did the default config
<Ursinha> lifeless, no, the script sets that
<rstat1> So its definitely an apache issue
<lifeless> Ursinha: does or should
<Ursinha> lifeless, should
<lifeless> Ursinha: how should it ?
<Ursinha> that's what you're supposed to change
<Ursinha> lifeless, identifying a [rollback] tag in the commit message
<lifeless> ok, let me look at that stuff
<lifeless> Ursinha: how are you generating your test data for test_commit -
<Ursinha> lifeless, in sample_data.tar.bz2 there's a branch
<lifeless> :(
<lifeless> you might like to migrate to self.make_branch_and_tree; its much more convenient IMO
<lifeless> Ursinha: how do you propose rollback will mark what commit its rolling back ?
<Ursinha> lifeless, when a person submits a branch that is a rollback, it marks the commit message as such
<lifeless> I get that
<Ursinha> that's adding [rollback]
<Ursinha> ah
<lifeless> but how do you propose we determine *what* branch is being rolledback
<Ursinha> lifeless, sorry
<lifeless> if you said
<Ursinha> lifeless, by bug number
<lifeless> but the rollback doesn't have one itself, does it ?
<lifeless> we have two commits, A and B
<lifeless> A says 'bug X'
<lifeless> B says 'rollback'
<lifeless> according to the wiki page
<Ursinha> if a branch/bug is stuck with qa-needstesting or qa-bad, and the next commit is a rollback, you allow that last stuck revision to be rolled out
<lifeless> we can look for tags in bug X when we're looking at commit A
<lifeless> Ursinha: it won't be the next commit though
<lifeless> Ursinha: it will be many commits - the pqm queue depth - later
<Ursinha> lifeless, hm, right, so we would need to guarantee that all the others in between are ready to land
<Ursinha> hm...
<lifeless> Ursinha: well thats a separate problem.
<Ursinha> I recall discussing that on the epic
<lifeless> Ursinha: here's what I think
<lifeless> developers should land a rollback branch
<lifeless> and manually mark the bug as qa-rollback
<lifeless> the rollback commit *itself* gets treated as deployable
<Ursinha> right
<lifeless> so if we have three commits
<lifeless> A - faulty
<lifeless> B - ok
<lifeless> C - rollback
<lifeless> when we process them, we see
<lifeless> A:-> bug X -> tagged rollback
<lifeless> B:-> bug Y -> tagged qa-ok
<lifeless> C:-> rollback
<lifeless> and we let all three past in this case
<lifeless> if C has landed but bug X is still tagged qa-bad, we assume that C was rolling back something else.
<lifeless> what do you think?
<lifeless> we can look at a fancy way to figure out what C rolled back later
<Ursinha> reading
<lifeless> but right now we have a many branches with revnos that differ etc, so I'd be worried about adding a revno to the rollback message, or a bug.
<lifeless> OTOH we could add [rollback=X]
<lifeless> to mean 'rolls back bug X'
<Ursinha> that was the idea, but indirectly, marking the commit as [rollback] and it having a bug linked
<lifeless> linking a bug to it would be hard today, as neither bzr nor lp support links other than 'fixes'
<Ursinha> lifeless, so your idea is to not depend on the script to identify which bug it's rolling back?
<lifeless> Ursinha: as a first iteration
<lifeless> Ursinha: I want to achieve 'ready to switch to this layout' _asap_; and improve once we're ready.
<Ursinha> lifeless, ok
<lifeless> spm: btw ping
<Ursinha> so we have rollback and bad in the same bug, or only one of them?
<Ursinha> tags, that is
<lifeless> Ursinha: I think for sanity we should only have one qa-STATE tag at a time
<lifeless> if its bad, we're blocked on a matching rollback
<Ursinha> right
<lifeless> hmm
<Ursinha> that was my understanding
<lifeless> actually - ah, got it.
<lifeless> so, I propose we say [rollback=REVNO-ON-DEVEl]
<lifeless> it is more work than I'd like
<lifeless> but if we don't do that, we'll deploy something broken, I guarantee it.
<spm> lifeless: yo
<lifeless> and when we set qa-bad we need a revno too
<lifeless> Ursinha: what time is it for you ?
<Ursinha> lifeless, 9h24pm
<lifeless> spm: heya. Staging prod shema QA environment
<lifeless> Ursinha: ok, I'll write up a proposal for you
<lifeless> and mail it to you and mars.
<Ursinha> lifeless, are you doing this right now?
<lifeless> I need to play on staging a bit and see if what I have in mind will work, and I don't want to keep you up late just waiting on me :)
<spm> lifeless: I'll need a little more context than that :-)
<Ursinha> lifeless, because I'm merging two branches to trunk that would be better for you to write code on top of them
<lifeless> Ursinha: I'll deal
<lifeless> if you want me to work on em, land em now :)
<Ursinha> lifeless, okidoki
<lifeless> spm: rt 40482 or something like that
<lifeless> spm: yes, thats the one
<lifeless> spm: we've missed you for a week, I wants to monopolise you now :)
<spm> ....
<lifeless> welcome back ?
<spm> it's good to be 'needed'; for values of 'need' :-)
<lifeless> :)
<lifeless> Ursinha: basically what I'm going to arrange is enough data flow that commits in devel that are messed up are persistently recorded as messed up
<lifeless> Ursinha: and commits that fix messed up ones record in their commit message that they do that
<Ursinha> lifeless, right
<lifeless> Ursinha: is it ok with you if I just edit the wiki page and ask you, Diogo and mars to tell me if I messed up ?
<Ursinha> lifeless, that's ok, we have history there :)
<lifeless> Ursinha: we'll need to change the lp-land stuff for rollback
<lifeless> to take a parameter
<Ursinha> lifeless, I was expecting that
<lifeless> ok cool
<Ursinha> lp-land and ec2 land
<lifeless> Ursinha: ok, updated the page: qa-rollback state is gone
<lifeless> Ursinha: please see if it makes sense :)
<lifeless> spm: ok, so jocularity aside: are we there yet ? :)
<jam> wgrant, lifeless: of course now I'm getting "Couldn't find a distribution for 'zope.publisher==3.12.0'" *sigh*
<wgrant> jam: update your download-cache.
<wgrant> Or just run rocketfuel-get, which does all that for you.
<jam> wgrant: well, through a mix of various voodoo incantations, it finally works. Thanks for your help
<jam> (rocketfuel-get isn't quite enough because it assumes you have an unpacked 'devel' working tree, etc. but it seemed to do what I needed)
<Ursinha> lifeless, seems ok
<rockstar> thumper, around?
<rockstar> I needs a query run on staging: http://pastebin.ubuntu.com/479695/
<rockstar> lifeless, maybe you can help? ^^
<rockstar> spm, ^^
<spm> rockstar: not without the magic word
<rockstar> spm, even on staging?
<rockstar> Oh, er, please?
<spm> hahaha
 * rockstar is American, so forgets his manners often...
<spm> rockstar: actually, the magic word, so SWMBO tells me: is **NOW**!!. fwiw...
<rockstar> spm, oh, then I should have gone with my instinct.
<rockstar> :)
<spm> heh
<spm> rockstar: 0 rows
<rockstar> spm, boo...
<lifeless> Ursinha: thanks
<lifeless> Ursinha: nearly done
<Ursinha> lifeless, I'm feeling ill, I'll talk to mars and matsubara-afk tomorrow to be sure I'm not missing anything :)
<lifeless> Ursinha: go sleep!
<rockstar> spm, can you please try http://pastebin.ubuntu.com/479698/ **NOW**!!
<thumper> rockstar: whazzup?
<spm> rockstar: same; 0 rows
<rockstar> thumper, I just needed some SQL queries run.
<rockstar> spm, and how close to production is the staging db?
<wgrant> Doesn't staging have its build queue cleared during the restore?
<rockstar> wgrant, ooh...
<spm> wgrant: along with several other tables of 'do stuff', I'd hope so.
<rockstar> spm, can lifeless give the okay to run the first query on production?
<lifeless> I can
<lifeless> thumper should by the new policy in preference to me, as hes your tl
<thumper> lifeless: I could but I'm multi-tasking with non-laptop tasks
<rockstar> lifeless, yeah, but thumper's got his hands full.  :)
<lifeless> ok
<spm> rockstar: aiui, we only need approval to run modification queries; selects are fine. happy to be corrected.
<lifeless> for now its mods that must get approval and be logged as incidents with high pri bug about not repeating it
<rockstar> spm, could you run the first query on production then?
<spm> rockstar: see. there's that missing magic word again. tsk tsk tsk.
<rockstar> spm, **NOW**!!
<spm> rockstar: 86 rows; need a paste?
<spm> heh
<rockstar> spm, not yet.  Lemme get a better query first, one that actually tells me more.
<spm> 85... <== it's going down.
<rockstar> spm, wait, what?
 * rockstar is confused
<rockstar> wgrant, around?
<rockstar> spm, point of clarification: you ran http://pastebin.ubuntu.com/479695/ correct?
<wgrant> rockstar: Indeed.
<wgrant> Why wouldn't it be going down?
<wgrant> You're looking for unstarted jobs.
<lifeless> sometimes b-m builds stuff
<mars> lifeless, nice performance tuesday follow-up mail, especially the steps you went through while solving.  I learned a thing or two from it.
<lifeless> mars: cool, thanks
<wgrant> lifeless: Occasionally.
<lifeless> of course, now I'm fighting all sorts of shitty aqlobject-half-transition pain
<spm> rockstar: hrm. no. was using the '698 paste, but doing so with 695.
<wgrant> lifeless: Well, you could always port the problematic callers to something that isn't ancient.
<spm> rockstar: and 78 rows now.
<rockstar> spm, okay, I'd be interested to know if that one changes.
<rockstar> (I suspect it won't)
<lifeless> wgrant: have a look at BugTaskSet.search() and its callers
<lifeless> wgrant: (e.g. yes, I want to)
<wgrant> lifeless: Looks Stormy to me already...
<lifeless> return SQLObjectResultSet
<lifeless> in what possible world is that storm ? :)
<lifeless> wgrant: its got some bits one way and some another - and its /callers/ are depending on some particular behaviours
<lifeless> wgrant: its nuts.
<wgrant> lifeless: Ah, true.
<wgrant> Missed that bit.
<spm> rockstar: 73 now... :-)
<rockstar> spm, oh crap...
<wgrant> rockstar: This must be the first time I've heard someone complain about buildd-manager being too fast.
<thumper> rockstar: need to talk about it?
<rockstar> wgrant, I'm complaining about the fact that my queries are proving my hypothesis wrong, and I don't like square one.
<thumper> spm: is staging up?
<rockstar> thumper, possibly.  One more query.
<lifeless> thumper: end of an upgrade cycle I think
 * thumper waits
<spm> thumper: I suspect not. looks like we're in the security.py part of a DB restore/setup
<thumper> spm: ETA?
<rockstar> Oh wait, crap.
<spm> "RSN"
<spm> it's been going for around 1h 45 atm. so soonish.
<lifeless> thumper: are we there yet. ""
<rockstar> thumper, yes, I'd like to talk about it, if you have some time.
<thumper> spm: I know RSN means NFI but soon
<thumper> ish
<thumper> rockstar: sure
<spm> thumper: damn. you're onto me....
<wgrant> rockstar: What's going on?
<rockstar> wgrant, investimigating the cause of https://bugs.edge.launchpad.net/launchpad-code/+bug/618860
<wgrant> rockstar: What's the traceback?
<wgrant> That suggests that either the actual or estimated start time are wrong.
<wgrant> s/wrong/None.
<wgrant> Neither of which your query will identify.
<rockstar> wgrant, http://pastebin.ubuntu.com/479708/
<wgrant> rockstar: So you're searching for WAITING with a NULL date_started. Every waiting build should satisfy that. Don't you really want to look for non-WAITING with NULL date_started?
<rockstar> wgrant, *facepalm*
<wgrant> In fact, you probably want RUNNING.
<rockstar> spm, can you please run http://pastebin.ubuntu.com/479709/ on production **NOW**!!
<lifeless> sadness, /participants on staging still timesout
<spm> lifeless: staging may not be up atm... is doing a DB restore/setup
<spm> rockstar: woo. 3 rows.
<rockstar> spm, okay, great.  Let's wait a bit, I'll tweak the query, and we'll run that again to see if it changes.
<wgrant> What's their status? 1?
<lifeless> spm: I know
<lifeless> spm: the frontend is back
<lifeless> whats the oops sync freq for staging ?
<spm> it was 3 min at one point...
<lifeless> sigh
<lifeless> 1691S10
<lifeless> when you have a cycle.
<spm> 6 minutes
<lifeless> hmm, I probably asked at 5 minutes...
<lifeless> its there
<lifeless> thanks
<lifeless> SQL time: 10011 ms
<lifeless> -sql time: 73 ms
<lifeless> Total time: 10084 ms
<lifeless> Statement Count: 8
<lifeless> \o/ select count(*) ...
<lifeless> OTOH 8 queries is fairly good
<lifeless> so yeah, its finished the updated, still timing out :P
<lifeless> probably we want something more sophisticated than decoratedresultset here
<lifeless> because - count(*) doesn't need all the left joins
<lifeless> (OTOH count(*) there is a bad idea anyway)
<mtaylor> thumper: "Compare with another revision" in loggerhead doesn't seem to do what I was assuming it would do
<mtaylor> thumper: that being - to compare this revision with another revision
<thumper> mtaylor: loggerhead has issues
<thumper> mtaylor: but it does work, but the ui for it is hard to understand
<mtaylor> thumper: ok. :)
<lifeless> \o/ 5.8 seconds to do the count(*) :<
<mtaylor> lifeless: that's not good
<lifeless> I'll paste you the query
<lifeless> mtaylor: its not entirely unexpected
<lifeless> we have some attributes that are 1:M and we're selecting the first-one in correlated subqueries for a team with 174 folk in it
<lifeless> denormalasing and caching the preferred email and default archive would save a batshit amount of time, I suspect
<lifeless> or
<lifeless> it may be bad indexes or something on some component
<lifeless> I'm pulling sections of the query out and its staying the same ;)
<lifeless> also, I bet sodium just shat itself
<lifeless> so, back to qa tagging
<lifeless> spm: please time http://pastebin.com/CV4wDPE1 on a prod slave
<spm> lifeless: 8.5 seconds
<lifeless> spm: grah
<lifeless> spm: need to find out why
<spm> and 2 x 6 seconds on repeats.
<lifeless> when sodium comes back I'll pokey pokey there
<spm> lifeless: https://pastebin.canonical.com/35966/ fwiw....
<lifeless> oh god I hate views
<lifeless> have I mentioned that?
<lifeless> can probably make this fractional-second without that view in there
<spm> i do str you mentioning a mild dislike of views a few weeks back....
<wgrant> Which view is it today?
<lifeless> validpersoncache
<wgrant> Ah, ValidPersonCache?
<lifeless> which is a LIE
<wgrant> is that really a view?
<lifeless> a stinking lie
<wgrant> It's not a view in the dev schema.
<wgrant> Or.
<wgrant> Wait.
<wgrant> It *is* a view, not a cache. That's right.
 * wgrant headdesks.
<mwhudson> !
<thumper> heh
<wgrant> rockstar: How is it Fix Released?
<wgrant> It was broken just an hour ago...
<thumper> mwhudson: how do you specify a layer for the create_initialized_view?
<thumper> mwhudson: what form does the layer param take?
<thumper> mwhudson: class? or string?
<rockstar> wgrant, that's the glory of running sql to fix the bug.  :)
<mwhudson> you can probably haul what the view is checking into the rest of the query?
<wgrant> rockstar: But isn't there still a bug that created the broken data?
<rockstar> wgrant, the corrupted records were from before we added the cancel build button, and we were cancelling builds with raw sql.
<wgrant> Ah, right.
<wgrant> Sorry, I'm just a little wary about people waving away buildfarm data corruption bugs without explanation.
<mwhudson> thumper: um, probably class
<wgrant> That tends to come back to bite us and cause things to burn down.
<mwhudson> thumper: i'd lean to passing a request that provided the appropriate layer though, i think
<thumper> mwhudson: attempting to pass the CodeLayer through
<thumper> lets see...
<thumper> I've been a bit more strict on where the views live
<thumper> now all the tests fail :)
<rockstar> wgrant, yeah, I'm not about to sweep buildfarm bugs under the road.
<thumper> yep that works
<mwhudson> thumper: haha
<mwhudson> good though
 * thumper ndos
 * thumper nods even
<lifeless> SELECT COUNT(*) FROM Person JOIN TeamParticipation ON TeamParticipation.person = Person.id LEFT JOIN KarmaTotalCache ON KarmaTotalCache.person = Person.id LEFT JOIN PersonLocation ON PersonLocation.person = Person.id LEFT JOIN Archive ON Archive.owner = Person.id LEFT JOIN EmailAddress ON EmailAddress.person = Person.id LEFT JOIN ValidPersonCache ON ValidPersonCache.id = Person.id WHERE TeamParticipation.team = 238131 AND NOT (Te
<lifeless> bah
<lifeless> anyhow
<lifeless> sorry
<lifeless> 19ms without validpersoncache
<lifeless> :)
<lifeless> mtaylor: ^
<lifeless> wgrant: ^
<thumper>  LEFT JOIN ValidPersonCache ?
<thumper> I thought you said without?
<wgrant> lifeless: I wonder how hard it is to drop that view.
<mtaylor> lifeless: lovely
<lifeless> thumper: as I said, paste fail
<lifeless> thumper: thats not the query without it
<lifeless> without it, and with the logic put back in via account directly
<lifeless> its 32ms
<lifeless> LEFT JOIN account on person.account=account.id  WHERE TeamParticipation.team = 238131 AND NOT (TeamParticipation.person = 238131) AND (Archive.id IS NULL OR Archive.id = (SELECT MIN(Archive.id) FROM Archive WHERE Archive.owner = Person.id) AND Archive.purpose = 2) AND (EmailAddress.status IS NULL OR (EmailAddress.status = 4 AND account.status=20));
<lifeless> is the end of the updated query
<wgrant> lifeless: How does the view make the plan so bad?
<wgrant> Also, EmailAddress.status IS NULL?
<lifeless> dunno
<wgrant> WTF?
<lifeless> wgrant: teams
<wgrant> status is NOT NULL.
<wgrant> Ah, right.
<wgrant> Fair point.
<wgrant> But I'd rather you checked Person.owner explicitly.
<lifeless> so, this isn't *quite* right, we're losing 4 participants somewhere, but I know the general shape that needs changing
<lifeless> wgrant: if you want to do this follow on fix, I'd be delighted
<lifeless> oh, I know why, I need an ACCOUNT.status is NULL in there
<lifeless> wgrant: you have to be very careful otherwise you exclude rows you don't want to
<lifeless> wgrant: or generate a torturous query plan
<lifeless> wgrant: semantically though, status is NULL IFF no row was returned
<lifeless> thats the correct end bit : AND (account.status is NULL or account.status=20)));
<lifeless> so you'd need a pretty deep nested check on person.owner to do it your way.
<wgrant> lifeless: I know. But you don't want to check that there's no emailaddress; you want to check that it's a team.
<wgrant> Why would it need a deep check?
<lifeless> so
<lifeless> consider this:
<lifeless> AND (EmailAddress.status IS NULL OR (EmailAddress.status = 4 AND account.status=20));
<lifeless> assuming we don't have data corruption
<lifeless> mmm
<lifeless> let me get back to you in a sec
<lifeless> you are suggesting
<lifeless> AND (person.teamowner IS NULL OR (EmailAddress.status = 4 AND  account.status=20));
<lifeless> right ?
<lifeless> to replace the whole email address and account status checks
<jtv> hi folks
<spm> heya jtv
<lifeless> wgrant: the problem is that that generates 663 rows, not 174
<wgrant> lifeless: What.
<jtv> hi spm
<wgrant> Something's not doing what we think it is, clearly.
<jtv> whoo-hoo, FakeLibrarian just landed
<wgrant> lifeless: What are the two complete queries?
<wgrant> And what is their purpose?
<lifeless> wgrant: see the pastebin above for the current one in devel
<lifeless> for /participants
<lifeless> Person._all_members
<lifeless> wgrant: http://pastebin.com/1iQZhpSJ is a hand updated one to discard the view
<lifeless> spm: ^ could you time that on a prod slave please?
<spm> lifeless: meh. try again. only 30ms on the first run; 15ms on the repeats. still too slow.
<lifeless> sorry
 * spm can spot a lack of sincerity at 100 paces
<lifeless> we're going to spend 10 times that parsing into python
<lifeless> so yeah
<lifeless> its hard to be concerned
<spm> :-D
<wgrant> lifeless: I wonder if we can just delete Person.archive and hope nobody notices.
<lifeless> wgrant: quite possibly, but note that we're delivering it pretty darn quickly
<wgrant> Yes, but it's awful.
<lifeless> wgrant: sure, I'll let soyuz folk worry about that :)
<lifeless> its simpler for me to just make it fast than worry about whether folk are using it.
 * wgrant stabs.
<thumper> mwhudson: I think I've found a problem :)
<thumper> canonical_url has a bug
<lifeless> sigh
<lifeless> sodium, you suck
 * jtv bates breath until thumper elaborates, and is now turning purple
<thumper> gah
<thumper> canonical_url(some_branch, view='+edit') now fails
<thumper> because the edit view is only on the CodeLayer
<lifeless> spm: could you try http://pastebin.com/ev56h4eY ? I would on staging, but devpad is dead
<thumper> even though the canonical_url(some_branch) specifies the code subdomain
<lifeless> and we don't have access to sourc
<thumper> not sure how to fix this one...
<lifeless> isn't that what curtis was talking about yesterday?
<lifeless> in his review of one of jc's things?
<thumper> lifeless: I don't know
<thumper> do you remember which review?
<lifeless> sorry no
<lifeless> but he was saying that our same-host url logic was rather strict
<lifeless> or something like that
<lifeless> sounds very similar
<lifeless> spm: ^
<mwhudson> thumper: um
<mwhudson> thumper: canonical_url looks at the current request
<thumper> mwhudson: yeah
<thumper> mwhudson: any idea how to create a request based on the rootsite?
<mwhudson> are you calling canonical_url while a request is 'going' in your tests?
<mwhudson> thumper: LaunchpadTestRequest() i hope?
<mwhudson> if not
<mwhudson> request = LaunchpadTestRequest()
<thumper> there is probably an anonymous interaction going
<mwhudson> directlyProvides(LaunchpadLayerOrWhatever, request)
<thumper> hmm...
 * thumper goes to make a coffee to help think
<jtv> thumper: can I get some?
<jtv> lifeless, I was just wondering about another side of canonical_url that might have interested you yesterday.
<jtv> It basically interates a singly-linked list of ICanonicalUrlData(foo).inside.
<jtv> Then for each of those, it gets ICanonicalUrlData(foo).path.
<lifeless> O(N^2)
<jtv> But it's essentially a recursion: walk from "here" to the root of a tree.
<lifeless> or worse
<lifeless> jtv: yes, I think its rather terrible :)
<jtv> I don't see the nÂ² part there tbh
<jtv> â¦but why not cache the cumulative paths to cut short the iterations?
<lifeless> well, things get renamed
<jtv> This is all browser code.  It only live for the duration of the request.
<lifeless> so a good case to consider
<lifeless> 'rename this branch
<jtv> Not much renaming going on in a GET, and not much URL rendering going on in a POST
<lifeless> start of the request, its foo
<lifeless> end of the request its bar, and we issue a redirect to the canonical url
<lifeless> so, its cachable, just have to invalidate
<lifeless> does canonical_url show up in profiling for us ?
<jtv> well, the link formatter does.
<lifeless> what does kcachegrind think is using up the time
<jtv> I don't know, tbh; I'd have danilo & adi who looked into a page where the link formatter was a deadly source of delays.
<jtv> Link formatters and menuâ¦ thingies were the big ones IIRC
<jtv> And I'm guessing the same principles apply.
<lifeless> so, I'm open to this sort of thing; theres no reason many things can't be cached; just need an invalidation strategy - and to understand *why* it needs a cache.
<thumper> jtv: sure, fly here and I'll make you a coffee
<jtv> lifeless: right, imagine a branch listing for a project.  Each branch link will probably repeat the same steps for the same project or person.
<lifeless> yes
<lifeless> if thats cheap, doesn't matter
<lifeless> if its not, it does
<jtv> thumper: the best part of that idea is, I think I can get a decent coffee at the airport on the way
<lifeless> I'm not saying I'm for or against, I'm saying lets get the data
<jtv> lifeless: no worries, I'm just adding some illustration I happened to have at hand before I get off my figurative arse to get data.
<jtv> lifeless: I do remember one thing, i.e. that MenuAPI was definitely a big killer there.  Very costly.
 * jtv wonders if just turning some of its hugely expensive-looking @property's into @cachedproperty's would be enough to shortcut repetitive computations
<thumper> ââ¹
<jtv> thumper: don't stick it in your ear, it's bad
<jtv> 8< :-(
<thumper> jtv: no shit sherlock
<thumper> mwhudson: I can't really use a LaunchpadTestRequest for the canonical_url fix
<thumper> mwhudson: what I really need to do is to create a valid request for the base url part of the canonical_url
<thumper> or should I say, the urlparts
<thumper> or rootsite, which is valid by the time we need to create a request
<mwhudson_> yay flaky wifi
<thumper> mwhudson_: can I skype you to talk this through?
<mwhudson_> thumper: ok
 * thumper waits for mwhudson_ to come online
<mwhudson_> oh yeah
<mwhudson_> thumper: i'm online now
<lifeless> spm: ping
<lifeless> spm: could you please apply 11370 from lp:~lifeless/launchpad/registry onto staging's appserver? its a teeny tiny patch
<lifeless> mwhudson_: pear seems rather uhm, unhappy
<mwhudson_> lifeless: context?
<mwhudson_> hardware -> losa
<mwhudson_> software -> a launchpad-code developer
<mwhudson_> >:)
<lifeless> mwhudson: cron mails
<mwhudson> lifeless: i don't get them any more
<lifeless> ah
<lifeless> File "/srv/importd.launchpad.net/production/launchpad/cronscripts/code-import-dispatcher.py", line 46, in <module>
<lifeless>    script.lock_and_run()
<lifeless>   File "/usr/lib/python2.5/xmlrpclib.py", line 787, in close
<lifeless>    raise Fault(**self._stack[0])
<lifeless> xmlrpclib.Fault: <Fault -1: 'OOPS-1691XMLP15'>
<wgrant> mwhudson: Don't worry, you are forever branded as a Code developer :P
<lifeless> with a ... in the middle
<mwhudson> lifeless: that's a serverside oops
<mwhudson> lifeless: it's a timeout
<mwhudson> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1691XMLP15
<lifeless> SQL time: 8125 ms
<lifeless> bit worrying
<lifeless> that it sat around for 8 seconds after that before trying another query
<lifeless> also worrying that it took 8 seconds to issue an update
<lifeless> possibly just another slow page killing it
<mwhudson> it's probably contention
<mwhudson> there was a short thread on the mailing list about this
<lifeless> a while back?
<mwhudson> just before the epic i think
<lifeless> you're updating the same page or something?
<lifeless> or the same rows
<mwhudson> yeah, same rows
<wgrant> Was it something about multiple jobs updating the machine heartbeat?
<wgrant> I forget.
<poolie> lifeless: you asked before about running multiple vms
<poolie> i don't think there's much to it is there?
<poolie> the best thing i learned was to use vm-builder
<poolie> as i described on https://dev.launchpad.net/Running/VirtualMachine
<poolie> and then there's just the sadly usual fluff of getting dependencies set up properly
<poolie> losa ping, could you please hurry up my builds in https://edge.launchpad.net/~bzr/+archive/proposed/+packages
<wgrant> You know, it would be interesting to have a graph of build farm queue depth.
<wgrant> And build farm throughput.
<spm> poolie: did you have a priority of "really wants" there? cause that's about 5 mins of clicky pain you're inflicting on me
<poolie> urk, really?
<poolie> in that case never mind
<spm> click the page; open the build; click to arch; click to edit score; edit score; save; repeat. fun fun fun. :-/
<spm> wgrant: I think we do... or something similar at any rate
<lifeless> spm: hi
<lifeless> spm: 16:12 < lifeless> spm: could
<lifeless> bah
<lifeless> 16:11 < lifeless> spm: ping
<lifeless> 16:12 < lifeless> spm: could you please apply 11370 from lp:~lifeless/launchpad/registry onto staging's appserver? its a teeny tiny patch
<spm> lifeless: yo, that patch should have just finished restarting
<lifeless> thanks!
<lifeless> \o/
<spm> works?
<lifeless> yeah
<lifeless> Total Bytes	159653 bytes
<lifeless> Request Timing	@0ms for 2510ms
<lifeless> Response Timing	@2510ms for 851ms
<lifeless> still slower than I'd like, but well within my stage 1 perf targets
<lifeless> spm: you can back that out now, if yo ulike
<spm> lifeless: is it problematic if it stays and just gets wiped in the next staging update?
<lifeless> fine by me
<spm> cool; I'll leave it in then
<lifeless> of course, it hasn't been qa'd etc yet
<poolie> lifeless: i'm getting repeated timeouts copying packages, is that known?
<lifeless> yes
<wgrant> poolie: Copy fewer each time.
<wgrant> And hope that it works.
<poolie> yes, that's what i was going to do
<poolie> just wondered if it was already known
<wgrant> yes, the copy checker is slow :(
<wgrant> The copy itself is tiny.
<thumper> mwhudson: thanks, but I think we're done
<mwhudson> thumper: i think so too
<mwhudson> frigging wif
<mwhudson> i
<lifeless> https://code.edge.launchpad.net/~lifeless/launchpad/registry/+merge/32953
<lifeless> meh
<wgrant> spm: Do you know if there's a ticket about setting up the PPA log parser on germanium?
<stub> spm: We have a choice of 1) upgrade sourcherry to Lucid with both PostgreSQL 8.3 and PostgreSQL 8.4 installed and do a test migration and switch staging to PostgreSQL 8.4, or, 2) Do a test migration from PostgreSQL 8.3 on sourcherry to PostgreSQL 8.4 on another box, then upgrade sourcherry to Lucid and PostgreSQL 8.4
<lifeless> evening stub
<stub> Morning :)
<lifeless> stub: do you have any opinion on validpersoncache ?
<lifeless> stub: using it in /participants vs using account directly gave 6second queries vs 35ms ones
<lifeless> stub: (its a view now, even it if wasn't in the past)
<stub> I tend to consider it legacy - Storm makes things nicer so we don't need views as a crutch. However, they should perform pretty much identically as validpersoncache isn't doing anything your direct query isn't.
<stub> lifeless: Have you confirmed your join criteria matches validpersoncache's, or did you not need to join with emailaddress for instance?
<lifeless> stub: same row count
<lifeless> stub: https://code.edge.launchpad.net/~lifeless/launchpad/registry/+merge/32953 shows the transition
<lifeless> (same row count executing the sql by hand, the patch should get the same but I haven't hand-checked the generated sql yet)
<poolie> wgrant: there is a graph of build queue length in lpstats
<stub> So you have removed the check for a preferred emailaddress.
<poolie> not updated for a while
<lifeless> stub: no
<stub> lifeless: but I see emailaddress is joined earlier and filtered later..
<lifeless> stub: still check that - the query returns teams as well
<wgrant> poolie: Ah.
<poolie> i have a note here to make tuolumne more robust
<poolie> at the moment if any monitor method fails, nothing gets updated
<stub> I'll have a look at the function... I'm missing context
<lifeless> stub: https://lp-oops.canonical.com/oops.py/?oopsid=1691S10
<lifeless> shows the vpc based query
<stub> So yes, unpacking the view worth doing here as the emailaddress table may already be joined with
<stub> lifeless: The check for result.preferredemail concerns me though as I suspect it will do a database query. There is some magic there though, so maybe it is prepopulated.
<lifeless> stub: its prepopulated by the row before
<lifeless> decorator-thing-before
<lifeless> so its fine
<lifeless> and there is a test on the query count for the API
<stub> Ok. The query might be a little more efficient if you add 'EmailAddress.status == 4, EmailAddress.person == None' so you don't see invalid rows in the first place.
<lifeless> stub: the new one performs at 35ms for ubuntu-dev :)
<stub> Yes, and adding the extra filter might be even faster
<lifeless> stub: remember that it returns teams too, deliberately.
<lifeless> so we want those rows
<stub> So Or(Person.teamowner != None, And(EmailAddress.status == 4, EmailAddress.person == None))
<lifeless> stub: I agree that when it gets generalised to prepopulating any person query, that we'll want to conditionalise that on whether we expect teams or not
<lifeless> stub: for some reason, doing something very much like that cut off rows we want
<wgrant> person == None? Huh?
<stub> Which could be another bug.
<stub> An emailaddress might be linked to just an account, not an account and person. Checking for person == None is probably unnecessary.
<wgrant> Also, can we please shoot ShipIt in the face so we can delete this whole Account mess?
<lifeless> we can also switch to inner joins when we don't want teams back
<stub> wgrant: +1
<wgrant> It is the only reason all this crap still exists.
<lifeless> wgrant: is *that* where it comes from?
<stub> Oh right... its an outer join for teams
<wgrant> lifeless: Not where it comes from, but why it still exists.
<stub> But my expression should be fine for that
<lifeless> stub: yes, which is why comparing a field needs a (field is NULL or field=xx) otherwise all the NULL rows disappear
<lifeless> even inside an OR, for reasons I don't remember right now
<stub> lifeless: There would be a 1:1 relationship between person and account except for shipit, as you don't need to use Launchpad to use shipit.
<lifeless> I wonder if shipit could move onto the SSO
<lifeless> and just use LP for karma checks
<stub> lifeless: EmailAddress is only checked if Person.teamowner is NULL, so only for people.
<stub> lifeless: ISD was going to rewrite it in Django, but I don't think it is scheduled.
<lifeless> I'll dig up the exact queries and debug with you later
<stub> lifeless: But what you have is already an improvement.
<lifeless> stub: separate from a rewrite, I mean :)
<lifeless> stub: ok with me landing it? jtv has reviewed
<stub> lifeless: Could do, yes. Simplest to give it its own database.
<stub> lifeless: sure
<lifeless> \o/
<wgrant> lifeless, stub: If we can convince SSO to send karma in the OpenID response, or ShipIt to use the API for karma checks, we can just run ShipIt off in its own little DB with a static copy of the LP codebase.
<stub> lifeless: I'm +1 on expanding all our views.
<stub> urgh. And here I am thinking shipit was already separate from the LP tables except for Account and EmailAddress. Karma too :-P
<wgrant> It also uses TeamParticipation, but SSO provides that.
<wgrant> So it's not much of a blocker.
<lifeless> stub: thats why I said 'lp for karma checks' :)
<lifeless> API for karma checks makes a great deal of sense to me
<lifeless> wgrant: want to patch it ?
<lifeless> wgrant: I bet a patch to the pre-breakout code would apply.
<lifeless> stub: care to note the expanding-views thing in dev.lp.net/Database/Performance ?
<wgrant> Heh.
<lifeless> any us folks still around?
<lifeless> would like to know how long https://api.staging.launchpad.net/1.0/~ubuntu-dev/participants takes for you (before staging updates blow it away)
<wgrant> 0.9s to wget it.
<lifeless> hmm, nice
<wgrant> edge is still waiting.
<lifeless> yes, it will be
<wgrant> Oh.
<wgrant> No, it's only 1.0s from edge.
<lifeless> hmm
<wgrant> First negotiation just took a while. Not sure why.
<lifeless> theres something wrong there
<wgrant> Might be cached, since I'm anonymous.
<lifeless> yes
<lifeless> this exact url is hit by some client
<lifeless> showed up in oops reports a while back, I put my branch together but was blocked on cachedproperty and rollouts n stuff
<wgrant> 2.6s for staging uncached.
<lifeless> tolerable
<wgrant> production ~7
<spm> wgrant: yeah there is - to re-enable it. but it needs some manual love to verify it doesn't go bananana's on us again.
<wgrant> What's required for that manual love?
<lifeless> wgrant: prod and edge are still running the pre-refactored code
<lifeless> wgrant: which means their count(*) is on the teamparticipation table only
<spm> stub: I'd be in favour of option 1; if it breaks hard, we learn. worst case, recover from backup (wheee)
<lifeless> and then you're seeing the lookup time for 50 people only
<spm> wgrant: *time*
<wgrant> spm: Ah :(
<lifeless> I really want to look at a kcachegrind of it
<lifeless> because theres just no justification for it taking more than a second ><
<lifeless> I mean
<lifeless> 35 ms + 35ms = 70ms, so wtf is the time going
<wgrant> From right next to the DC, it takes 80ms for a cached response, ~2s uncached.
<wgrant> Oddly slow.
<wgrant> Sometimes up to 3.5s...
<stub> spm: Ok. I should RT this or discuss on losas@ ?
<spm> stub: probably email; it's kinda a funky one.
<lifeless> that fix is ec2ing
<lifeless> poolie: by multiple vms I meant
<lifeless> poolie: glue to run 1/8th per vm, or whatever
<poolie> oh, no, i haven't done that
<poolie> wbn
<poolie> i have one vm running the whole suite, one vm for actual development, and then a host OS running a browser and at 2mb/vm that's about it
<lifeless> stub: thanks
<lifeless> Ursinha: present for you: https://code.edge.launchpad.net/~lifeless/qa-tagger/qa-rollback/+merge/32959
 * wgrant mutters something along the lines of "why is the QA tool private?"
<lifeless> wgrant: told you already
<lifeless> wgrant: and will be fixed, but not top of the pops
<wgrant> Oh, right, I remember now.
<wgrant> Sorry.
<lifeless> you know what would be neat
<lifeless> the revisions list in mps and branch pages linking each rev with an expander
<lifeless> like on loggerhead
<wgrant> You mean having a code browser not completely unintegrated with LP?
<lifeless> that would show the lh list of changes in the rev
<lifeless> and that clickable to expand into a full diff
<lifeless> should be easy thanks to mwhudson jsonifying the guts
<lifeless> .count is storm, right ?
<wgrant> Yes.
<lifeless> man this is a mess
<lifeless> (I'm back on bug search)
<wgrant> At least you only have to fix it once.
<lifeless> doubtful
<lifeless> this is going to take stabs and stabs
<lifeless> also
<lifeless> search
<lifeless> great name in a typed language
<lifeless> fffffreaking painful in python
<poolie> spm, if you're still here, can you add or check my gpg key
<spm> poolie: oh sorry - I completely forgot about that from this morn. 1001 apologies.
 * spm chases...
<spm> poolie: actually I'm not going to get to that. I'm about 8 levels deep in critical fail interrupts atm. afaict, the prob is you don't actually have access in the config; irrespective of the key. suggest an RT is in order; it's relatively easy to do.
<poolie> spm, no problem at all, it's not critical
<poolie> mostly just wondered if i was doing it wrong, as often happens :)
<poolie> is it just me or did lp lose all its icing?
<wgrant> poolie: See #launchpad.
<wgrant> edge update broke things.
<poolie> i see
<adeuring> good morning
<lifeless> poolie: reminder (assuming I didn't miss it) - talk up your awesome flags stuff to the list
<poolie> lifeless: my last thing for today is to bump the docs along a bit then post
<poolie> then cook a pumpkin, etc
<lifeless> \/ pumppkins
<lifeless> its scary how tolerant storm is of different ways of spelling shit
<poolie> maybe i'll reorder that
<poolie> crap, poopie, poop, poo, ...
<lifeless> I have a bastard hybrib frankensteins monster of sqlobject & storm ... and its still working
<bigjools> morning all
<wgrant> Morning.
<poolie> lifeless: ok, mail sent, please followup with anything else you thing ought to be said about it now
<lifeless> poolie: done and thanks.
<lifeless> this is huge ;)
<poolie> will pydoctor interpret rest correctly?
<poolie> or something else?
<poolie> actually what i mean is, what markup can i use in docstrings?
<lifeless> I've just been doing the bzr thing
<lifeless> and assuming I'll fix it up later
<wgrant> I think pydoctor does ReST. But I'm not sure what the Zope apidoc thing wants.
<poolie> lifeless: good question in mail
<mrevell> Hallo
<poolie> hi mrevell
<poolie> mrevell: francis suggested you might help me do a UDD user surevy
<poolie> to help us get some data on what to do next
<wgrant> That initialism makes me sad.
<poolie> ?
<lifeless> abbrev. perhaps ?
<wgrant> UDD means Ultimate Debian Database.
<lifeless> anacronynm
<StevenK> wgrant: Just as well they didn't call it Debian Ultimate Database
<mrevell> poolie, I'd be happy to. Were you thinking something Surveymonkeyish?
<lifeless> hey
<lifeless> is store = getUtility(IStoreSelector).get(MAIN_STORE, DEFAULT_FLAVOR)
<lifeless> what Class.select(...) does in the sqlobject compat layer ?
<poolie> yes
<lifeless> what is the recommended store lookup means
<lifeless> the one that uses slaves for gets
<wgrant> lifeless: IStore(SomeClass)
<bigjools> IStore(blah)
<lifeless> thanks guys
<wgrant> You can also say IMasterStore(SomeClass) if you really want a master object.
<bigjools> you can force IMasterStore(class) or ISlaveStore(class)
<lifeless> why would someone use StoreSelector ?
<bigjools> old skool
<wgrant> lifeless: Because IStore wasn't invented yet.
<poolie> are fixtures already an lp dependency, or likely to be soon?
<mrevell> poolie, I can either give you access to the LP team's paid Surveymonkey account or if you give me an idea of what you want to know, I'll be happy to set something up for you and then get your approval before I publish it.
<poolie> the second sounds like less work (for me :)
<mrevell> :) No problem. Do you want to email me with what you'd like to know?
<StevenK> bigjools, wgrant: I'm using IStoreSelector).get(MAIN_STORE, MASTER_FLAVOR) in InitialiseDistroSeries, should I fix that?
<poolie> mrevell: so handwaving, because i have to stop soon
<wgrant> StevenK: Yes.
<wgrant> StevenK: it's deprecated. Kill it if you see it anywhere.
<poolie> we just want to know: did they use it, did they hit problems, if so where, is it any better than what they're doing now, what would make it so, are there any particular niches where it is good, bad, potentially but not actually good,
<wgrant> StevenK: I was guilty of using it when I started. I'm not sure if IStore didn't exist yet, or whether it just wasn't known commonly enough.
 * bigjools wonders if IStore() will work when we get more stores.
<wgrant> bigjools: Why would we have more stores?
<wgrant> Unless we're completely rearchitecting everything.
<bigjools> scalability
<StevenK> wgrant: So, IStore(self) ?
<wgrant> StevenK: I think so.
<bigjools> is just one reason
<wgrant> Or maybe just Store.of(self). Not quite sure.
<mrevell> poolie, Thanks for that. I'll work something up this (my) morning and email you with a link to the draft survey.
<wgrant> StevenK: But you could just IMasterStore(class you're looking for)
<bigjools> StevenK: IStore() needs a class, not an object
<poolie> wow
<poolie> lazyness pays off :)
<wgrant> bigjools: But we can't do multi-master...
<poolie> thanks
<bigjools> wgrant: we *could* given the motivation
<wgrant> bigjools: Could we?
<wgrant> I mean, is it actually possible?
<wgrant> I don't think we can do multi-master without splitting the DB.
<bigjools> correct :)
<wgrant> At which point we are changing everything.
<mrevell> np @_)
<mrevell> er, didn't mean to do the psychadelic smiley there...
<bigjools> wgrant: if we add more publishers we're going to need to scale writes
<bigjools> anyway, otp
<wgrant> The publisher does just about no writes!
<wgrant> Just a crapload of reads.
<wgrant> Unless you're publishing a fresh new copy of the archive or series.
<wgrant> I guess.
<wgrant> But that's rare.
<bigjools> think about deathrow etc
<wgrant> deathrow doesn't do too much writing either. But it does a lot more stuff that it doesn't need to do.
<wgrant> I get the feeling that nobody has really touched it since it was first written.
<wgrant> Because some of what it does is just wrong.
<wgrant> Like trying to remove every binary publication until the last publication of that binary is gone.
<wgrant> Similar with sources.
<wgrant> And not removing overridden files.
<lifeless> wgrant: BugTaskSet.search is freaking nasty to convert
<wgrant> And just generally being pathetically inefficient in various other ways.
<lifeless> I don't suppose you know of a parser for sql order by constraints
<wgrant> lifeless: Howso?
<wgrant> No, why, and will it make me sick?
<lifeless> *yes, it takes SQL as an input*
<wgrant> Oh, right, that one.
<wgrant> Yay.
<wgrant> You can't pass that into Storm directly?
<lifeless> sure
<lifeless> once you find every users of BugTaskSearchParams that sets orderby
<wgrant> :(
<lifeless> exactly
<lifeless> hmm
<lifeless> perhaps outputting a warning and using the storm helper...
<StevenK> lifeless: I've forgotten your Storm notes. :-( Select -> compile, and then what?
<noodles775> Hi mrevell... I'm just finishing the diff UI changes at https://dev.launchpad.net/LEP/DerivativeDistributions and bigjools has already updated the 'Creating a new derived distroseries' mockup, so I think you could start organising the next round when you've time.
<lifeless> StevenK: compiler(expr, State())
<lifeless> bah
<lifeless> compile()
<lifeless> mumble
<lifeless> look at store.execute for inspiration
<wgrant> Do you actually need to manually compile it?
<lifeless> eventually no
<StevenK> lifeless: Yes, I've done the compile(), it's what I do after that
<lifeless> first cut yes
<lifeless> StevenK: you should have a string now. execute it
<lifeless> wgrant: needs an InsertInto Expr with compile-hook to DTRT
<lifeless> StevenK: if your string doesn't have INSERT INTO ... SELECT FROM
<lifeless> StevenK: then do a single simple string insertion at this point
<lifeless> wgrant: the built in on in storm wants VALUES, not SELECT
<lifeless> s/on/one/
<wgrant> Ah.
<mrevell> wonderful, thanks noodles775
<lifeless> wgrant: not hard to whip one up. ETIME, Diminishing returns.
<lifeless> hmm
<poolie> it seems to want epytext
<bigjools> wgrant: lamont enjoyed* rolling back the buildd package yesterday
<lifeless> whats the storm equivalent for SQLConstant?
<bigjools> have a look in storm.locals
<lifeless> or rather, maybe I should say
<lifeless> its SQL
<lifeless> what I actually mean though
<lifeless> is
<lifeless> is 'sqlobject.sqlbuilder' coming from storm, or sqlobject
<lifeless> like, do we still have sqlobject around?
<lifeless> if this works
<lifeless> I'm going to be a) happy , b) very very very very scared
<bigjools> oh awesome, I can't search for numbers in bugs, it thinks it's a bug number
<lifeless> theres a bug on that :P
<lifeless> try +filebug
<bigjools> heh
<wgrant> bigjools: yeah, so I heard :/
<bigjools> I used to use the dupe finder for searching all the time
<wgrant> It wasn't QA'd on DF first?
<bigjools> wgrant: we don't have any amd64 builders on DF
<wgrant> Ah. Yay.
<lifeless> bigjools: why did you stop ?
<bigjools> lifeless: it stops after so many bugs
<bigjools> although I'd really, really like some of the "advanced" options on the first search page
<bigjools> and I'd like the default page to present on newness like it always used to
<wgrant> bigjools: I missed it because I did some tiny cleanup after an amd64 test, and then my amd64 machine became unavailable so I only tested the final trivial changes with i386/lpia. :/
<lifeless> bigjools: you might like to try again, with the relevance changes etc, *might* get what you want higher up
<lifeless> bigjools: I'd like that too
<wgrant> And managed to get the arguments around the wrong way in the process, which only breaks amd64-on-amd64 builds :(
<lifeless> bigjools: I have the feeling that some of the 'remove X its slow' we've done are treating symptoms not causes and can be put back in.
<bigjools> wgrant: shit happens :(
<bigjools> lifeless: yeah
<wgrant> Anyway, the builds are all retried and we'll QA the fix it on dogfood today.
 * bigjools can't wait for a fast LP
<lifeless> hey its coming
<wgrant> It's coming more rapidly than I expected.
<bigjools> wgrant: so that 404 bug you moved to Soyuz, it's a dupe but I can't find the bug it's duping
<bigjools> lifeless: \o/
<wgrant> bigjools: That's what I said.
<lifeless> wgrant: what di dyou expect?
<wgrant> bigjools: I know there's a bug about it.
<wgrant> At least one.
<wgrant> But I cannot find it.
<bigjools> lifeless: I expected nothing less from you :)
<wgrant> lifeless: Um, not immediate timeout decreases and massive speedups that you're achieving.
<wgrant> I was expecting more inertia.
<lifeless> wgrant: I've removed a couple of small roadblocks I think
<lifeless> wgrant: but I know *everyone* wanted it faster
<lifeless> so I really can't take much credit
<lifeless> hell, its taking me days to land each of my patches
<lifeless> learning curves etc
<lifeless> wgrant: bigjools: Thanks though, its nice to be blamed for good stuff :)
<bigjools> lifeless: it's more about bringing the impetus and the culture change.
<wgrant> lifeless: You may not be entirely to blame, but things certainly seem to have been happening a lot more since you arrived.
<wgrant> Right, what bigjools said.
<bigjools> I've wanted a culture change for ages, but a lot of devs seem happy to throw stuff out there without thinking about performance
<wgrant> Gah.
 * bigjools used to work on stuff that *had* to perform well
<wgrant> To make matters worse, bug #404 is private.
<bigjools> lol
<wgrant> (yes, yes, I just ran into that trap myself without thinking)
<bigjools> I can't believe I just search for "missing link"
<lifeless> :)
<bigjools> I found a long-lost ancestor :)
<wgrant> I still can't find the bug.
<bigjools> me neither!Â¬
<wgrant> I looked a couple of hours ago when the guy appeared in #launchpad.
<bigjools> I know it's there
<wgrant> I thought it had 404 in the summary.
<wgrant> But maybe not.
<lifeless> is there an existing 'take the first element of this tuple' adapter around ?
<lifeless> itemgetter.duh
<wgrant> I was about to say.
<bigjools> thing[0] ? :)
<lifeless> closures
<wgrant> bigjools: So, I thought I'd found it. But then I realised it was the one filed a few hours ago.
<wgrant> I am mystified as to the location of the old one.
<bigjools> if you find it, let's change its title so we can find it again (and the dupe finder can find it more to the point)
<lifeless> one dupefinder issue
<lifeless> - it does not search comments
<wgrant> bigjools: There's bug #522800. I thought there was another, but I may be wrong.
 * wgrant kicks the bot.
<lifeless> \i/
<bigjools> _mup_: bug #522800
<wgrant> However, your analysis isn't quite correct.
<wgrant> The old file is expired, which is arguably right.
<wgrant> The problem is that getFileByName returns the oldest matching LFA, when it should probably returned the oldest unexpired.
<bigjools> yeah I suspected I might be wrong
<wgrant> Each upload has its own LFA.
<wgrant> With the same LFC.
<wgrant> I think.
<lifeless> ok, time to see the damange with this storm sqlobject frankenquery
<lifeless> see you all tomorrow
<bigjools> nn
<wgrant> Night.
<wgrant> bigjools: Thanks, you added the term we couldn't search for :P
<bigjools> wgrant: the dupe finder gets it nicely ;)
<wgrant> True.
<bigjools> "broken" is the word I missed when looking before
<jml> it takes far too long to get an EC2 image ready to run LP tests.
<noodles775> james_w: Hi! I've just been updating some of the derived distro mockups, but had to make some assumptions while doing so, and I'd like you to check that they're valid.
<noodles775> In particular, with: https://dev.launchpad.net/LEP/DerivativeDistributions#Derived%20series%20differences
<noodles775> For various reasons (details in the comments that follow the mockup), I've made the assumption that you'll only want to add a comment on a row when you mark it as ignored... is that an invalid assumption, or ok?
<wgrant> Oh wow, this looks awesome.
<noodles775> Thanks wgrant! (or are you talking about something else :) )
<wgrant> I'm talking about this.
<wgrant> Hopefully we'll eventually be able to make Ubuntu a derivative of Debian...
<noodles775> Yeah, that'd be exciting :D
<wgrant> Then I can kill off mdt.
<deryck> Morning, all.
<wgrant> thumper: Hm, I thought Translations did some layer-specific views.
<benji> gmb: a question came up in a review of one of my branches yesterday that I want to run by you; the MP is at https://code.edge.launchpad.net/~benji/launchpad/bug-580035/+merge/32917
<benji> gmb: the question is: the branch adds a section to a doctest, is that acceptable to you guys?
<gmb> benji, I think so. deryck can give you an authoritative answer though.
<deryck> is it a doc or a test? :-)
 * deryck looks at MP
<benji> it's more testy than docy
<deryck> benji, yeah, I'm sorry to be a pain, but I would prefer this be a unit test.  Especially given two things -- we're trying to get doc tests that are tests converted to unit tests and we're especially doing this around email and subscriptions currently.
<deryck> my rule of thumb, if it's primarily a test, it should be a unit test.  A doc with testable parts is, of course, okay.
<benji> I'll convert it then.
<deryck> benji, thanks!
<jml> hmm hmm hmm.
<james_w> noodles775: hi, have you looked at merge-o-matic?
<noodles775> james_w: Is that what normally displays at https://merges.ubuntu.com/main.html ? If so, yep. Otherwise no.
<noodles775> what in particular have we missed?
<james_w> noodles775: yeah, that's the site. I just wondered if you had looked at the sort of comments people left there
<james_w> my belief is that they leave them for things such as "I'm working on this", or "This depends on foo", in which cases you wouldn't hide the row
<james_w> noodles775: the mockups are looking great though, thansk
<noodles775> james_w: no, I hadn't looked at the comments. Right... I'll update then, thanks.
 * jml lunches
<Ursinha> bigjools, I'm confused, is bug 618231 really won't fix?
<bigjools> Ursinha: yes
<bigjools> check the diff for the patch that was landed :)
<Ursinha> bigjools, I wonder why that branch mentioned the bug :)
 * bigjools shrugs
<jml> \o/
<jml> just locally reproduced a failure I previously had to spin an ec2 instance to get
<henninge> bac: Is the case ignored when imports are sorted?
<henninge> Aaa, Bbb, aaa
<henninge> or
<henninge> Aaa, aaa, Bbb
 * henninge looks in ... somewhere
<henninge> hm, one example in PythonStyleGuide suggests aaa, Aaa, Bbb
<jml> henninge, I think that example is true to form
<jml> henninge, c.f. https://dev.launchpad.net/EmacsTips
<henninge> jml: it's a shame though that "sorted" sorts the other way round ...
<jml> henninge, sorted(imports, key=lambda x: x.lower())
<henninge> hm, that will simply ignore case
<jml> henninge, anyway, if you are changing *everything*, then I think you get to make up the new rule :)
<henninge> oh, cool! ;-)
<henninge> since that seems to be the natural sorting order in Python ...
<jml> I'm ok with the change. But do let people know, so dev scripts can be updated.
<jml> Python sucks.
<jelmer> jml: What's it done wrong this time?
<lifeless> existing?
<cody-somerville> lol
<jml> jelmer, why is "lambda: foo.append(None)" acceptable but "lambda: count += 1" not?
<lifeless> expr, statement
<lifeless> hysterical raisins
<jml> lifeless, yeah, I know the expr / statement thing.
<jml> of course, lambda: foo.__iadd__(1) is ok
<jml> get your act together, Python
 * jelmer tries hard not to mention the H word
<jml> jelmer, I know the word you're thinking of, of course
<jml> jelmer, but I'd reach for Lisp before that if I were looking for a language that gets this right.
<jml> ffs, even Javascript gets it right.
 * jelmer wonders if vala does closures
<lifeless> jkakar: hi
<jml> lifeless, if I were to write something that made graphs based on subunit stream, to which project should I submit it?
<lifeless> subunit or tribunal come to mind immediately
<jml> OK. I'm ec2 landing my branch which completely changes ec2test/remote.py and as such is likely to break other people's "ec2 test" experiences
<lifeless> cool
<lifeless> did you find a reviewer ?
<jml> I'll try to send out an email when it hits devel, but in case I don't and weird things start happening, at least there'll be an IRC record.
<jml> lifeless, yeah, jelmer reviewed it.
<lifeless> \o/
<jml> once it lands & is on stable, I'll then move on to the bit where I deliberately change the behaviour.
<lifeless> thank you for staging it
<jml> np.
<jml> any other way and it will only end in tears.
<jkakar> lifeless: Hiya
<lifeless> jkakar: would like your input on https://bugs.edge.launchpad.net/launchpad-foundations/+bug/618019
<lifeless> jkakar: I think perhaps gustavo hasn't understood our concerns
 * jkakar looks
<jkakar> lifeless: I see.
<jkakar> lifeless: I would like to see time-to-fetch measured, probably as a separate measurement to time-to-execute-statement.
<jkakar> lifeless: I sometimes wonder if I'd also like to see time-to-instantiate-objects, since I *believe* the cost there is non-trivial.
<jml> jkakar, ime, .values() is much faster
<jkakar> jml: Yep, exactly.
<jkakar> At that point Storm is just generating a query and then handing you almost the exact result received from the Python DB driver.
<lifeless> jkakar: so, do you know why gustavo is pushing back here?
<lifeless> jkakar: it astounds me
<lifeless> deryck: hey
<bac> henninge: sorry, i didn't see your question.  yes, existing coding standards say case-insensitive sorting (is that what you'd call it?) but we have lots that are not.
<deryck> hi lifeless
<jkakar> lifeless: I'm not sure I understand either.  Have you talked to him directly about it?
<lifeless> deryck: are you aware of any reason why stormifying BugTaskSet.search would be a bad idea?
<lifeless> deryck: other than it being hard because its big :)
<lifeless> jkakar: not yet
<deryck> lifeless, I'm not aware of any reasons against.  I suspect it wasn't done only because of the amount of work.
<lifeless> so SQLObjectResultSet hates DecoratedResultSet
<lifeless> guess what thats lead me down the path to do...
<deryck> ah, ok.  Sure, I see no problem with doing this.
 * jkakar needs to learn about DecoratedResultSet since it seems to get mentioned a lot.
<jkakar> I don't know what it is but I wonder if whatever it does shouldn't be in Storm proper.
<lifeless> jkakar: have a look
<lifeless> ./lib/canonical/launchpad/components/decoratedresultset.py
<lifeless> jkakar: it lets you do
<lifeless> DRS(store.find((Foo, Bar, Baz)), adapt)
<lifeless> where adapt is
<lifeless> def adapt(row):
<lifeless>     person = row[0]
<lifeless>    person.friends = row[1]
<lifeless>   person.has_geolocation = len(row[2] > 0)
<lifeless>   return person
<lifeless> so it becomes a result set yielding persons with stuff setup about them
<jkakar> lifeless: Oh, very nice.
<lifeless> Ursinha: hi
<Ursinha> lifeless, you just read my mind
<lifeless> :)
<lifeless> <- talented
<Ursinha> hahaha
<Ursinha> lifeless, what do you intend with the bad-commit-xxx tag?
<Ursinha> is that a replacement or a complement for qa-bad?
<lifeless> complement
<Ursinha> (or needstesting, because a rollback can also be a rollback for a untested revision)
<lifeless> we have to remember that the *commit* is bad even when the bug becomes qa-ok
<lifeless> we can't *ever* rollout that commit without its rollback=xxx commit
 * lifeless wants a whiteboard
<Ursinha> lifeless, that's ok, understood
<lifeless> Ursinha: so
<lifeless> Ursinha: where does this leave us
<Ursinha> lifeless, branch is merged
<lifeless> like
<lifeless> if we said 'we're switching to this workflow now', what would be missing
<lifeless> *) A QA environment for it
<lifeless> anything else?
<lifeless> mthaddon: still around ?
<Ursinha> lifeless, now, production_stable is a branch that has merges from db-stable
<Ursinha> I'm thinking aloud
<lifeless> Ursinha: that fine :)
<Ursinha> we'd need a QA environment, right
<lifeless> I'm wondering if we could change staging to use stable
<lifeless> and change it back for release week
<Ursinha> that would be our "stable"
<Ursinha> hmm
<Ursinha> lifeless, problem is staging is where people test db changes now (I guess)
<Ursinha> because it's db-stable, basically
<Ursinha> so we'd need another QA environment for that
<Ursinha> wait
<Ursinha> lifeless, where do you intend to have db changes landing to be tested?
<lifeless> staging
<lifeless> :)
<Ursinha> lifeless, so I don't get it :)
<lifeless> ok
<lifeless> have you seen my most recent mail to the list - just this morning
<Ursinha> ah, no
<lifeless> flacoste: this may interest you as well
 * Ursinha looks
<lifeless> Ursinha: I'm proposing a temporary thing by reusing staging
<lifeless> Ursinha: long term we definitely want two environments
<lifeless> flacoste: ^ thinking this might reduce the work needed by losas before we can *start* RFWTAD
<Ursinha> lifeless, so, stable would be deployed to staging
<Ursinha> with db and no-db changes
<Ursinha> and if qa-go-ahead, revision is also deployed on prod
<Ursinha> right?
<lifeless> I don't parse your second line, so I'll rephrase it
<lifeless> 'with a copy of the prod db and only schema changes contained in stable itself'
<lifeless> and yes
<Ursinha> "only schema changes contained in stable itself"
<lifeless> yes
<lifeless> like
<Ursinha> where else the schema changes would be?
<lifeless> someone might land a db patch that would have no downtime
<Ursinha> hm
<lifeless> like a brand new table
<lifeless> as per RFWTAD
<lifeless> flacoste: now I have your attention
<lifeless> flacoste: quick call ?
<flacoste> lifeless: sure
<benji> deryck: do you mind looking at my doctest to unittest translation (http://bazaar.launchpad.net/~benji/launchpad/bug-580035/revision/11319) and -- if you like it ;) -- commenting at https://code.edge.launchpad.net/~benji/launchpad/bug-580035/+merge/32917
<deryck> benji, looking now....
<benji> thanks
<deryck> benji, looks great.  Thanks for doing this work!  I'll update the MP now.
<benji> thanks much
<deryck> And with that, I'll EOD. :-)
<deryck> Later on everyone.
<lifeless> Ursinha: just chatted with francis
<lifeless> Ursinha: my idea was not taking enough consideration of the the time spent doing non-db dev on db-devel
<lifeless> Ursinha: so we'll still be blocked on the existing RT ticket
<lifeless> Ursinha: but thats ok, I'm increasing my concern for db agility as a result :)
<lifeless> flacoste: thanks :)
<Ursinha> lifeless, ok, I was wondering that too... but that's ok now
<Ursinha> I'll keep soketesting what we have
<Ursinha> and grab a coffee now :)
<thumper> sinzui: can you rename lp.registry.enum to lp.registry.enums ?
<sinzui> I can
<thumper> sinzui: it matches our directory naming, and will be consistent with lp.code :)
<thumper> sinzui: thanks
<sinzui> you mean right now, stop writing this boring report and do something to feel good?
<thumper> sinzui: great induction sprint write up
<thumper> sinzui: I'll be grabbing ideas from it
<thumper> sinzui: I'm not going to say that you should do it this instant, but ASAP would be good
<sinzui> thumper, We need to update the wiki with a sane example bazaar.conf and locations.conf. jcsackett had trouble working that out
<thumper> sinzui: where do you think it should go?
<thumper> sinzui: I could look to edit it
<thumper> sinzui: somewhere off the dev wiki I guess
<sinzui> I think so
<sinzui> There used to be stuff like that in  the old wiki
<lifeless> thumper: ping
<lifeless> sourcepackagename.path - I'm getting an error with that having 'distribution' set to None
<lifeless> in its repr()
<wgrant> sourcepackagename.path?
<wgrant> That doesn't exist.
<lifeless> sorry, sourcepackage.path
<lifeless> I get confused easy
<wgrant> That really shouldn't exist either, but it does.
<wgrant> A SourcePackage is a SourcePackageName in a DistroSeries.
<wgrant> So it's not valid to have one without a Distribution.
<lifeless> right
<lifeless> and an invalid one is being repr() in a traceback
<wgrant> Hmm.
<lifeless> making the traceback blowup
<wgrant> I wonder how that was obtained.
<lifeless> and thus being hard to debug
<lifeless> BugTaskSet.search misbehaving
<lifeless> that much I know, because, well, I'm overhauling the damn thing
<lifeless> distribution is itself a property
<lifeless> via distroseries
<wgrant> Hm.
<wgrant> So people can no longer subscribe to Ubuntu's bugs. Great.
<lifeless> to the whole feed ?
<wgrant> Yes.
<lifeless> feel free to weigh in on the MP / bug - I did
<thumper> lifeless: pong
<lifeless> they can subscribe to the list that gets all the mail
<lifeless> thumper: nvm, sorry - was teh sourcepackage thing above
<lifeless> but I'm just going to change the repr
<wgrant> lifeless: Yes, but surely making the UI suck less is a better general solution than just disallowing it for distributions.
<thumper> lifeless: I would like a chat with you sometime about the project cloud
<lifeless> thumper: sure.
<lifeless> thumper: anytime - now if you like
<thumper> lifeless: ok now, skype?
<lifeless> wgrant: this isn't a dichotomy
<wgrant> If people are doing things that only negatively affect them, permission changes are not the solution!
<wgrant> We need to work out why they are being stupid and remove the cause of the stupidity.
<wgrant> Not block the stupidity outright.
#launchpad-dev 2010-08-19
<lifeless> wgrant: so
<lifeless> wgrant: you're expressing the point I made
<lifeless> wgrant: but the point the malone folk made is also valid
<thumper> lifeless: advice: where should the canonical_url method go?  lp.app... ???
<lifeless> thumper: is it a function or a method ?
<thumper> lifeless: it is a function used all over the place
<thumper> right now it is in canonical.launchpad.webapp.publisher
<lifeless> so, I'd leave it there, for now
<thumper> but mostly imported from canonical.launchpad.webapp
<lifeless> probably we want lp.components or someting for it, but its not going to be cleaner just pulling it out
<lifeless> its fairly rightly connected to that core mess ;)
<thumper> what do you mean by lp.components?
<lifeless> oh bah
<lifeless> I wwas thinking of ./lib/canonical/launchpad/components
<lifeless> naming is hard. Call it chicken_url and be done :P
<mwhudson> thumper: we don't really have a place for most of the things in canonical.launchpad.webapp yet aiui
<mwhudson> thumper: of course, it should really be in lazr.something :-)
<thumper> :-(
<lifeless> thumper: so yeah, serious - don't move it as part of this, I know you want to, but its the wrong end of webapp to start at
<thumper> yeah
<thumper> :(
<thumper> I'm trying to get a failing test case
<thumper> translations have layer specific views for ITranslator
<thumper> but you can't go: canonical_url(factory.makeTranslator())
<thumper> nor for translation queues
<thumper> and no factory method for distro series language
<thumper> yah
<thumper> found one
<thumper> POFilee
<mwhudson> thumper: what do you need for a test case?
<thumper> an existing page on a specific layer
<mwhudson> is it possible to create that situation for a test?
<thumper> I found one anyway
<mwhudson> i wrote a test case that registered a new view a couple of weeks back
<thumper> ah
<thumper> remember where?
<mwhudson> lp.testing.test_publication i think
<mwhudson> it's bonkers
<mwhudson> but that can be hidden i expect
<mwhudson> lp.testing.tests.test_publication
<mwhudson> perhaps should be rewritten as a fixture rather than a test method
<thumper> mwhudson: heh
<thumper> mwhudson: you are right, it does look a little bonkers
<lifeless> mwhudson: you could use my shiny new project ;)
<mwhudson> lifeless: i need to look at that again
<mwhudson> lifeless: where is it?
<thumper> lifeless: another project?
<lifeless> lp:python-fixtures
<lifeless> or pypi
<lifeless> pyppi:fixtures
<mwhudson> lifeless: so the new thing over what testtools' useFixture is the reset() method?
<lifeless> mwhudson: its a more complete API, yes
<lifeless> also no tearDown
<mwhudson> oh ok
<lifeless> rather a cleanup contract
<mwhudson> well sure, if you have addCleanup...
<lifeless> so as a developer of these fixtures
<lifeless> you do different (and I hope less) work
<lifeless> I thought testtools had installFixture, rather than useFixture, or possibly even just LP has it
<mwhudson> oh right
<mwhudson> i was just going from http://pypi.python.org/pypi/fixtures
<lifeless> right
<lifeless> rest is nice :)
<mwhudson> hm
<mwhudson> i wonder how hard it would be to kill ILaunchBag
<jelmer> hmmm, lunch
<lifeless> I've love to kill the magic thread locals scattered all over
<lifeless> more context less magic. _please_
<mwhudson> lifeless: there's the zope interaction and the database connections, i can't see them going away
<lifeless> mwhudson: why not ?
<lifeless> both have the request as context
<mwhudson> well, for the former, i think it would be a pretty massive change to zope
<mwhudson> although maybe it wouldn't be too bad
<lifeless> I'm sure there would be fallout
<lifeless> but
<mwhudson> you're right though that the request is the real context
<lifeless> if we want to e.g. move to twisted throughout, or spread requests across threads in scatter-gather, or suspend requests for AJAX, we need to fix this.
<lifeless> and systematically, so that it doesn't bite us in the a**
<mwhudson> the launchbag particularly is stuff that's mostly semi-trivially derived from the request
<mwhudson> which is why it should be first up against the wall
<lifeless> so how does one order_by a Reference ?
<wgrant> I believe you need to join against it and order by a column of it.
<lifeless> ._local_key looks promising
<wgrant> The _ does not.
<lifeless> details
<lifeless> python, consenting adults. \o/
<wgrant> Ew.
<poolie_> hi all
<lifeless> ok, bbiab, selftest running
<lifeless> do we use __str__ in the UI anywhere ?
<lifeless> like, on distribution or such things ?
<lifeless> \o/ /participants fix in devel
<thumper> mwhudson: using flacoste's idea, I can make canonical url work without huge changes to the publisher zcml
<thumper>   <utility
<thumper>       component="lp.code.publisher.CodeLayer"
<thumper>       provides="zope.publisher.interfaces.browser.IDefaultBrowserLayer"
<thumper>       name="code" />
<thumper> getUtility(IDefaultBrowserLayer, "code") gets you the layer then
<thumper> s/"code"/rootsite/ and we should be good
<thumper> with an extra default for no defined layer
<mwhudson> thumper: yeah, makes sense
<thumper> is there a layer for the mainsite?
<mwhudson> canonical.launchpad.layers.LaunchpadLayer i think
<thumper> mwhudson: ta
<poolie> thumper: your thing of resolving lp: for private branches on the server side only
<poolie> how are you testing that?
<thumper> poolie: pushing to bzr+ssh://bazaar.staging.launchpad.net/+branch/project
<poolie> iwbn to eventually eliminate the upfront check in bzr
<poolie> at least for some cases, or at least experimentally
<thumper> poolie: yes, but there are several other things I want to get done first
<thumper> poolie: but yes, I agree
<thumper> poolie: and for the record, +lots from me on nested trees
<poolie> ok
<poolie> and i know commit-to-stacked is still around
<poolie> it may be small enough we can just do it
<thumper> +1 on that too
<mars> lifeless, https://devpad.canonical.com/~mars/qa-pipeline/deployable.txt
<lifeless> mars: looking good
<mars> I think it looks ugly, but it works!
<lifeless> its  alot quieter without the rrors
<lifeless> when its logged in it should be quite readable
<michaelh1> Hi there.  When is staging due back up?
<lifeless> anytime
<lifeless> it updates automatically
<lifeless> so its down when applying the updat
<lifeless> ehttps://staging.launchpad.net/successful-updates.txt
<michaelh1> It's been down for around 20 minutes.  Is that normal?
<lifeless> yes
<mars> https://staging.launchpad.net/successful-updates.txt for the link-lazy
<Ursinha> so everyone else is working late too :)
<Ursinha> mars, https://code.edge.launchpad.net/~ursinha/qa-tagger/change-to-newer-api/+merge/33078
<thumper> Ursinha: I'm not
<Ursinha> ha
<Ursinha> :)
<Ursinha> mars, it took me a bit more to submit that simple branch because that assignment wasn't working with my launchpadlib version; after talking to leonard I tried with the trunk version and it worked fine, then I submitted the mp
<michaelh1> Hmm.  Staging has been down for 90 minutes.  The last successful-update line is from 3 hours ago
<thumper> spm: ^^
<spm> awesome. it's the same failure that clobbered edge yesterday.
<thumper> michaelh1: hey
<thumper> michaelh1: regarding nested trees in bzr
<adeuring> good morning
<StevenK> Anyone around who can help cook some SQL?
<spiv> StevenK: stir-fry it with some mushrooms, garlic, and losa sauce.
<StevenK> Hah
<StevenK> spiv: And serve on a bed of Storm?
<spiv> StevenK: and scrambled python eggs.
<StevenK> The code in InitialiseDistroSeries currently grabs all of the packagesets in the packageset table associated with the parent distroseries and inserts them back, with the distroseries pointing at the child.
 * spm questions where spiv gets his 'losa sauce'. Does he have some sort for machine that crushes hapless losas!?!?! Said machine called "launchpad" or maybe "ubuntuone".
<StevenK> I'd like to do the same for the packagesetsources table, select all rows where the packageset is in the packageset table with a distroseries of the parent, and insert them back with the packageset ids pointing to the child
<spiv> spm: I don't reveal my sauces ;)
<spm> *groan*
<StevenK> Haha
<StevenK> This is complicated by the fact that there can be multiple packagesets for the distroseries
<mrevell> Alright everyone?
<noodles775> Yep :)
<mrevell> Good :)
<mrevell> noodles775, One of our participants in the second round is an Arm person working for Linaro.
<noodles775> Aha
<noodles775> mrevell: btw: I just updated two of the mockups this morning adding a comment column.
<mrevell> Ah, thanks noodles775
<stub> lifeless: Are you going to land https://code.edge.launchpad.net/~lifeless/launchpad/foundations/+merge/32299 ?
<stub> Or should I kick off an ec2 land?
<lifeless> stub: could you please
<lifeless> stub: I had thought it was going in with your branch :)
<stub> Its all been mushed together
<lifeless> stub: thanks
<lifeless> gnight
<deryck> Morning, all.
<jml> bac, mars: this is my rough plan for changing the ec2 mail: http://paste.ubuntu.com/480435/ â thoughts?
<jml> thoughts welcome from others, too.
<jelmer> jml: I'm not sure how hard having multiple attachments is, but would it make sense to attach the profiling information as an attachment as well.
<jml> jelmer, multiple attachments is pretty easy. which profiling data do you mean, and what would you use it for?
<jelmer> jml: The "Test suite profiling information"
<jelmer> jml: Actually, thinking about what I use it for, I don't need it.
<jelmer> I've been discovered a few launchpad layers I didn't know about using it, but that doesn't seem like a good reason to keep it in the emails.
<stub> sinzui: OpenIDIdentifier or OpenIdIdentifier?
<sinzui> I like the latter
<bac> jml: that sounds like a good plan.  i really like the idea of having failures up front.  makes checking it on a phone much better.  i don't do that often but flicking through an insanely long message to get to the bottom is tiresome.
<jml> bac, thanks.
<mars> jml, be careful with the stdout/stderr capture code.  if test_on_merge.py crashes, it writes to stderr.  Not sure if you can control stdout and stderr just for the tests themselves.
<jml> mars, good to know. I lean toward the "tests shouldn't print to stdout/err" approach anyway.
<jml> mars, fwiw, the way it works now is that subunit forwards all the gunk it doesn't recognize onto a file-like object that you give it.
<mars> jml, I should be more specific: if the test infrastructure detects a hang, it will start printing to the pipes, and that will be appended randomly to the output :/
<mars> jml, yep, the subunit forwarding would probably be enough.  Just wanted to make sure the output wasn't accidentally eaten.
<mars> jml, I like the outline for adding a sort of test summary to the head of the email (don't remember the exact terminology)
<jml> list of failing tests.
<mars> branches, revno, X passed, Y failed, roll of failures - all very nice
<mars> the plan looks good to me.  I would be happy with any of those changes
<mars> bac, why would you check the test failures on the phone?  Surely you can't *fix* the test failures on the phone? :)
<mars> If you can't do anything about it, why check?
<bac> mars:  b/c i'm nuts
<bac> mars: you know, you're out having a nice dinner, get email with a FAILURE message.  it's nice to know what failed so you can obsess over it rather than enjoying your meal.
<dobey> can anyone tell me what the problem is exactly in https://launchpadlibrarian.net/53996367/buildlog.txt.gz ? looks like bzr is not happy to build stuff on maverick with source package recipes?
<jml> I think that's a known bug, not being able to build stuff on maverick with source package recipes.
<jelmer> I think rockstar was working on a fix for that last week
<jelmer> s/I think//
<jml> https://bugs.edge.launchpad.net/launchpad-code/+bug/617072
<gary_poster> bug 607691
<gary_poster> _mup_ is dead :-/
<dobey> ah ok
<dobey> guess we just have to wait for it to get deployed to edge then
<niemeyer> bug 123456
<niemeyer> bug 123456
<niemeyer> Hmm
<niemeyer> bug 123456
<_mup_> Bug #123456: podcast crashes amarok <Amarok:Invalid> <xine-lib (Ubuntu):Fix Released> <https://launchpad.net/bugs/123456>
<niemeyer> _mup_: Way to go
<sabdfl> hi folks, could you amend the LP CSS to use UbuntuBeta as a preferred font, if it's available, please?
<sabdfl> and let me know when that's on edge?
<sabdfl> we'll have a Mono in due course too
<jml> sabdfl, will do.
<lifeless> gmb: love the quotes page quote..what was the context ? ;)
<sabdfl> thanks jml
<jml> sabdfl, np.
 * jml off for the day
<sabdfl> cheerio
<lifeless> danilos: flacoste: what bug was the cronscript db-loop-tuner issue filed as ?
<danilos> lifeless, none yet, sorry 'bout that
<lifeless> no worries
<lifeless> of course, I can't put my super clever simple fix in the bug until there is one :)
<lifeless> would you like me to write up my understanding as a new bug? .... it might be a bit biased
<lifeless> hah
<lifeless>         raise RuntimeError("Unexpected situation. "
<lifeless>     RuntimeError: Unexpected situation. Please contact the developers.
<lifeless> leonardr: nice work on the storm collection changes
<lifeless> leonardr: I'm really looking forward to seeing that live
<leonardr> thanks, it's still in progress...
<lifeless> naturally
<lifeless> jkakar: around ?
<leonardr> jamesh_, are you around for some storm help?
<leonardr> lifeless, i talked to jkakar today, he's on holiday and avilable intermittently
<lifeless> yeah, I was being hopeful
<lifeless> storm has decided to take a code path with an assertion in it :(
<lifeless> jamesh_ will be 2am local tz now
<leonardr> oh well
<lifeless> leonardr: maybe I can help ?
<lifeless> or were you asking for me ?
<leonardr> lifeless: no, i was just telling you about jkakar
<leonardr> i also need storm help
<lifeless> whats up
<lifeless> leonardr: whats the storm help you need ?
<leonardr> lifeless, i'm in a situation where i can call len() on a ResultSet object before getting the data, but calling len() after getting the data raises an exception
<lifeless> whats the exception
<leonardr> *** FeatureError: __contains__() does not yet support set expressions that combine ORDER BY with LIMIT/OFFSET
<lifeless> ah
<leonardr> limit and offset are somehow being associated with the dataset when i get the data, and it's persisting beyond the data fetch
<lifeless> so by getting the data, you mean slicing ?
<leonardr> yes
<leonardr> i kind of think it's a problem in batchnavigator
<lifeless> interestingly
<lifeless> my storm here does a copy() in __getitem__
<lifeless> I wonder if its a sqlobject glue issue
<flacoste> lifeless: good news, mthaddon said that they will be able to start on the new 'stable' staging
<leonardr> so does mine, that's why i think it's batchnavigator
<flacoste> iow, that doesn't need to wait for the completion of the Lucid upgrade
<flacoste> which should be finalized with the next DB roll-out
<lifeless> flacoste: \o/
<lifeless> flacoste: you da man
<lifeless> or rather
<lifeless> mthaddon and the losas are da men
<flacoste> the losa are the maen
<flacoste> exactly
<lifeless> :)
<lifeless> still
<lifeless> praise the messenger, shoot the messenger. Its all good :)
<lifeless> leonardr: whats the class of the resultset ?
<lifeless> leonardr: it might be a tableset, resultset or sqlobjectresultset,
<lifeless> leonardr: or a decoratedresultset
<leonardr> lifeless, pretty sure it's a resultset
<lifeless> leonardr: lets make certain
<leonardr> lifeless: ok, it's not a problem with batchnavigator. i can run len(self.list), and then i can slice self.list, and then len(self.list) raises an exception. self.list is the same object in all 3 cases
<leonardr> so it's either storm or the FiniteSequenceAdapter
<leonardr> now to answer your question
<leonardr> lifeless, self.list.context is a storm.store.ResultSet
<lifeless> whats self.list then, a security proxy ?
<leonardr> self.list is a canonical.launchpad.webapp.batching.FiniteSequenceAdapter
<leonardr> which does pretty much nothing--maps foo() to self.context.foo() for some common operations (iter, len, getitem)
<lifeless> ok
<leonardr> so it's probably not the FSA's fault
<lifeless> so does self.list.context.count(); self.list.context[whatever:whatever]; self.list.context.count()
<lifeless> also blow up ?
<leonardr> lifeless: no, that works
<leonardr> however, self.list.context[whatever:whatever].count() blows up
<leonardr> but i don't see where one object is being substituted for the other, even though i'm pretty certain that's what's happening
<lifeless> ah
<lifeless> []
<lifeless> the __getitem__ returns a curried resultset which only answers whatever:whatever
<lifeless> you aren't expected to call count on it
<lifeless> (and indeed, calling count on it would return a different value to count on self.list.context)
<leonardr> and afaict i never do. but i have a feeling that must be what's happening
<lifeless> does self.list.context.count() work after len(self.list) starts blowing up ?
<leonardr> i don't store the result of self.list[self.start:self.end+1]. the first thing i do is turn it into a python list, and i do store that
<leonardr> lifeless: no
<lifeless> leonardr: so once len(self.list) starts going boom, self.list.context.count() starts going boom
<lifeless> do you perhaps replace self.list.context?
<leonardr> checking now, but i don't think so
<leonardr> no, same object
<leonardr> unfortunately i don't have visibility into its ._limit
<lifeless> removeSecurityProxy ?
<salgado> james_w, in that thread about binary upload, bigjools said we no longer accept mixed uploads anywhere, but do we still accept binary-only uploads anywhere?
<leonardr> lifeless: that worked, but it makes things more confusing. both ._limit and ._offset are Undef
<lifeless> leonardr: are you in pdb ?
<lifeless> leonardr: if so, consider
<lifeless> debug self.list.context.count()
<james_w> salgado: from the buildds, unless they go through a different mechanism.
<leonardr> ok, give me a minute
<james_w> salgado: mixed upload is what we want in some cases for vostok.
<salgado> james_w, oh, right, I'd seen that in a test a few days ago and then forgot it completely
<leonardr> lifeless: i'm forming a hypothesis that the ResultSet object and the temporary copy of it have an 'expr' object in common
<salgado> james_w, yeah, that I got; was just trying to figure out if there were other things I could get rid of
<leonardr> and that this 'expr' object is the one that has its limit and offset modified by the count()
<lifeless> leonardr: that would be exciting
<lifeless> leonardr: and plausible
<lifeless> I'm facing a potentially similar exciting situation
<leonardr> lifeless: yes, the ResultSet self._limit is Undef, but self._get_select().limit is 3
<lifeless> \o/
<leonardr> lifeless: there's code in _get_select that is supposed to avoid this problem, but i do see a big "# XXX UNTESTED!" next to the relevant bit
<lifeless> -hah-
<lifeless> its bug filin time!
<leonardr> lifeless: ok, here's the problem
<leonardr> the expr object is shared between the slice and the original
<leonardr> _get_select includes code to change expr.limit if a new one is provided, but not to remove an existing expr.limit if one is _not_ provided
<lifeless> and that may be deliberate
<lifeless> ><
<leonardr> who knows
<lifeless> jaaaaamuuuuuuuu
<leonardr> so either _get_select needs to change or we need to copy the expr object
<leonardr> lifeless: https://bugs.edge.launchpad.net/storm/+bug/620508
<_mup_> Bug #620508: Slicing a ResultSet breaks subsequent len() calls. <Storm:New> <https://launchpad.net/bugs/620508>
<lifeless> I'd try changing it to fixup when limit isn't supplied
<lifeless> and see if any tests break
<leonardr> seems to work basically, but now that i've established it's not my fault i need to fight a different fire
<lifeless> gl
<salgado> james_w, do we still accept uploads without .orig.tar.gz files when they've been included in previous uploads?
<james_w> salgado: yes, that would be normal
<salgado> james_w, and in that case the .orig is included in the .dsc but not in the .changes, is that correct?
<james_w> salgado: correct
<james_w> and the checksum in the .dsc has to match the file we already have
<salgado> right
<salgado> james_w, I'm confused because there's a test which says it makes an upload where the .orig is missing (and indeed it's not in the .changes).  however, when I remove the .orig from the same directory where the .dsc and .diff.gz are, the test fails
<james_w> salgado: where's the test?
<salgado> maybe that's because I'm assuming it'd look up the .orig in the librarian but it's actually looking it up on that same directory
<salgado> james_w, lib/lp/archiveuploader/tests/nascentupload.txt
<salgado> it uploads lib/lp/archiveuploader/tests/data/ed_0.2-20_i386.changes
<salgado> if you remove ed_0.2.orig.tar.gz from there, the one test which says uploads without .orig are allowed breaks
<mtaylor> thumper: morning?
<mtaylor> lifeless: I need to report a bug, but it's on something I'm afraid will go away so I'm not sure how to capture/point folks at it
<lifeless> mtaylor: details ;P
<mtaylor> thumper, lifeless: https://code.edge.launchpad.net/~eday/nova/api-tests/+merge/32960
<mtaylor> lifeless: that's showing conflict markers as if there were a merge conflict
<mtaylor> lifeless: but if you merge it into the target, those merge conflicts are not there
<lifeless> so
<lifeless> its been merged already
<lifeless> naturally it won't conflict - theres nothing to do.
<lifeless> also
<mtaylor> hrm
<lifeless> you probably have your 'use weave merge' settings still
<lifeless> because lp doesn't have all plugins installed
<lifeless> and doesn't have per project tweaks
<mtaylor> I do? I don't have that set up at all
<lifeless> it cannot predict faithfully whats going to happen
<lifeless> hm
<lifeless> well
<salgado> james_w, yeah, the test is just lying, I think. it says it's not uploading the .orig bug in fact it makes it accessible to  NascentUpload directly, so it's like if it had been uploaded.  IIUC
<mtaylor> lifeless: so - it showed the conflicts markers as it does now - but when eday did a manual test it didn't conflict - so he marked it approved and it landed fine ... should I file a bug or should I wait until it happens again
<lifeless> uhm
<lifeless> it may have conflicted before trunk changes
<lifeless> we don't recalcualte the diff except when the branch itself changes
<mtaylor> ah
<mtaylor> ok. that explains it
<mtaylor> lifeless: I would like to request a "regenerate diff" button then
<lifeless> mtaylor: me too the bug then
<lifeless> I don't want a button
<lifeless> I want DTRT
<lifeless> regenerate the diff when you land on the page
<lifeless> or something
<mtaylor> well yeah, that would be great
<james_w> salgado: yeah, DSCFile will find the .orig.tar.gz in that path, even though it isn't in the .changes?
<mtaylor> I only want a button in as much as I want to be able to make sure I can do a current review
<salgado> james_w, exactly, but I think in a real upload with a .changes that doesn't include the .orig, it wouldn't be there -- it would have to be found in the distribution.  correct?
<james_w> salgado: I haven't dug that far. That would be the intention though
<mtaylor> while I'm whinging - it would be great if I could see all bugs with a certain tag regardless of project ... :)
<lifeless> mtaylor: patches ...
<james_w> salgado: yeah, DSCFile._getFileByName as part of DSCFile.checkFiles should get the LFA and compare the checksums
<mtaylor> lifeless: yeah - getting closer. todo list... long
<salgado> james_w, ok, I'll make a comment explaining that the test is cheating.  thanks for the help
<james_w> salgado: what's the failures
<james_w> salgado: I don't see why the test should have to cheat
<james_w> Is it "Unable to find %s in upload or distribution."? That would make sense
<salgado> james_w, well, it says the .orig is not included in the upload but it does include the .orig.tar.gz in the data that is given directly to NascentUpload
<james_w> ah, I see what you mean. Does the test fail if you delete that file?
<salgado> it does, yes
<mwhudson> gary_poster: ping
<gary_poster> hey mwhudson!  I have to run really soon, but can talk for a sec
<mwhudson> gary_poster: i wanted to talk a little about buildout
<gary_poster> sure
<mwhudson> gary_poster: for linaro we're open sourcing an internal project
<mwhudson> and a set up like launchpad uses for managing dependencies would probably work well
<mwhudson> but as with launchpad, the tendency of buildout/setuptools to go grabbing things off the internet is a bit undesired
<mwhudson> i seem to recall that you had to patch buildout to support this
<mwhudson> are those changes upstream yet>
<mwhudson> ?
<jml> hello again Launchpadders.
<mwhudson> also i remember you said you had a better plan that a branch for the download-cache, but i can't remember what it was now
<gary_poster> My changes are about being able to use a system Python
<mwhudson> oh ok
<gary_poster> They will hopefully be released next week in a beta 3 (a previous version was beta 1)
<gary_poster> You have two options for what (I thnk) you want
<gary_poster> 1) you can do the bzr-managed download-cache.  Kinda lame as you know, but gets you off the ground.
<gary_poster> 2) you can set up a random apache directory (mars is using devpad.canonical.com/~mars/pypi or something like that) to store the eggs you want and point the deployment to that directory using find-links.  If that's in the data center then you are good.
<gary_poster> Actually third option:
<mwhudson> gary_poster: what are the issues with using a system python?  trying to not use the stuff from that python's site-packages?
<mwhudson> gary_poster: also, one more question, one of the dependencies is a very large js library
<gary_poster> 3) you can use zc.sourcerelease for your own deployments.  This means that it goes over the internet when you build a package up to deploy.  Then you send the tgz over to where you want to deploy and run a provided script.  Ask mars for an example of how he is using it.
<lifeless> jml: why hello there
<mwhudson> gary_poster: would it be possible/not completely crazy to write a recipe that extracts the tarball and puts it somewhere so the code can find it?
<mwhudson> gary_poster: zc.sourcerelease means you basically create a tarball that already has the eggs/ directory populated doesn't it?
<gary_poster> system python: yeah, stuff in debian's site-packages is installed in a very different/conflicting way than other egg installations.  Lots of other niggly problems.  my patch lets you use system Python and still use some of the dependencies in it, or not as you wish.
<mwhudson> ok
<jml> I wonder if I should convert unexpected test runner errors into a result.addError call.
<jml> I think that will be simpler.
<mwhudson> i guess i need to try to understand that issue then
<gary_poster> very large js library: if it is packaged as Python library, np.  if it is packaged as a configure-make-make install, there's zc.recipe.cmmi.  dunno what to tell you about that
<mwhudson> ok
<mwhudson> i was a tiny bit surprised that there didn't seem to be a recipe for "untar this tarball *here*"
<gary_poster> write a recipe that does a tarball: yeah, sounds like zc.sourcerelease
<lifeless> jml: what does it do today?
<gary_poster> zc.recipe.cmmi might, not sure
<jml> lifeless, has a special logger method "error_in_testrunner", re-raises.
<mwhudson> gary_poster: ok, i'll take a look
<lifeless> ><
<mwhudson> gary_poster: did you consider other things like virtualenv + pip for launchpad?
<mwhudson> this is a whole space that i've kind of seen moving around from a distance for a long time, but not really paid any attention to
<jml> lifeless, before it just printed stuff to some files
<gary_poster> mwhudson, http://pypi.python.org/pypi/zc.recipe.cmmi has the string "tgz" in the docs ;-)
<lifeless> oh
<lifeless> that reminds me
<lifeless> time to mail barry a 'python packing is all wrong and heres what the stdlib has to do to fix it' note
<gary_poster> virtualenv + pip: perhaps not as closely as I should have, but yes.  virtualenv does not let you install eggs robustly with a system python.  It lets you make a virtual python with *no* system packages if you want
<gary_poster> but not letting some through
<gary_poster> which is what we wanted
<mwhudson> i think that's what we'll want too
<mwhudson> for psycopg2 and stuff like that
<gary_poster> I won't comment on pip, because I'd probably make an a** of myself
<gary_poster> right
<barry>  lifeless: pythons generally don't know how to pack, that's why they always arrive with way too much luggage than they need
<mwhudson> gary_poster: i at least hope to avoid having bzr branches in a directory called sourcecode/ :-)
<gary_poster> mwhudson: +1 :-)
<lifeless> mwhudson: what is the problem with that?
<gary_poster> mwhudson, I really have to release the new version of buildout.  I was on vacation last week and had a dental operation. this week.  I'll try to make the release next week.
<mwhudson> lifeless: well, having 2 mechanisms for managing dependencies is bad enough
<lifeless> mwhudson: agreed
<gary_poster> so that will help with the system Python thing if you go that way
<gary_poster> jml, do you happen to know if twisted.web.wsgi.WSGIResource is pretty cooked?  Putting that in front of Launchpad instead of ZServer seemed interesting
<gary_poster> I thought it might go a ways to giving an avenue for something lifeless was interested in
<lifeless> \o/
<mwhudson> long poll?
<lifeless> yeah
<gary_poster> yeah
<mwhudson> lifeless: we haven't completely ruled out making debian packages of everything either
<lifeless> mwhudson: that would rule
<mwhudson> but there are issues there too
<lifeless> mwhudson: but, there are some functional issues
<lifeless> hah
<jml> gary_poster, no idea
<mwhudson> like all the deps are already on the cheeseshop but not in ubuntu
<mwhudson> and currently rollouts don't require root
<gary_poster> debian packages issue: no good for people who aren't on debian
<gary_poster> rollouts don't require root: I'm pretty sure James would be happy to get rid of that requirement in exchange for getting everything packaged
<mwhudson> i hear rumours that using debian packages doesn't strictly *require* root for rollouts, but i don't know anything about them
<mwhudson> gary_poster: people who aren't on debian/ubuntu can always use a vm ;-)
<gary_poster> ack jml, thanks
<gary_poster> heh :-)
<gary_poster> ok guys I have to run
<mwhudson> i don't think there are any known problems with WSGIResource currently...
<mwhudson> gary_poster: thanks
<gary_poster> np :-)
<gary_poster> have a nice ...day or night
<mars> mwhudson, regarding zc.sourcerelease: it works pretty well.  It bundles the download-cache into the tgz, you run 'install.py' on the target system and it builds in place (download-cache items become eggs)
<barry> lifeless: yep.  mind if i forward your message to tarek and/or distutils-sig?
<mars> I am using it right now to run the latest versions of bzr, launchpadlib and httplib2 onto devpad (which still runs hardy)
<lifeless> barry: sure
<lifeless> barry: it may not need stdlib changes
<lifeless> barry: but it needs a champion in the debian community, one that can sensibly decide when something needs to be in-stdlib to be used, or just new-conventions
<lifeless> barry: for instance, one problem is that eggs are installed unversioned
<barry> lifeless: but eggs aren't used in debian packaging, right?
<lifeless> barry: they are
<lifeless> barry: dpkg -S egg-info
<lifeless> If I had more tuits I'd try just doing it
<barry> lifeless: oh i see what you're saying.  not the .egg format, but egg-infos
<lifeless> well
<lifeless> all the various bits of machinery
<lifeless> you could put eggs on disk
<lifeless> or you could use the version number to put unpacked egg dirs on disk and .pth files
<lifeless> or whatever
<lifeless> lots of options
<lifeless> the key thing is that
<lifeless> a simple 'import foo' needs to keep working
<barry> lifeless: thinking about this more, it's probably a debian-python policy thing.  and the trick will be in either setting up sys.path correctly, or adding import support in python (or a combination of both)
<lifeless> right
<lifeless> its -nearly- done
<barry> lifeless: i have to leave now (for an early rehearsal).  let's talk tomorrow or next week about it some more if you want
<lifeless> I don't want to talk about it... just want it working :)
<lifeless> But I'm happy to help-by-talking :P
<lifeless> -> new house ADSL connection, one hopes.
<wgrant> Gah.
<wgrant> Why do so many people want a server-side GitHub-like 'Fork' feautre?
<wgrant> This isn't Git, people...
<mwhudson> wgrant: because github is perfect, clearly
<jelmer> wgrant: because doing so is hard using the command-line in git..
<wgrant> jelmer: My guess is that it's because Git doesn't have stacking.
<mwhudson> more seriously, we probably need to work on documenting workflows with launchpad + bzr
<mwhudson> wgrant: i think it's because git push expects there to already be something git-ish on the remote side
<wgrant> mwhudson: Yeah, some documentation on code.launchpad.net/blah would be nice.
<wgrant> It'd also be nice if one could just push to lp:project without a branch being there already.
<mwhudson> thumper is working on that...
<wgrant> Yay.
#launchpad-dev 2010-08-20
<thumper> when I'm not trying to fix freaking annoying bugs that is
<mwhudson> thumper: canonical_url still?
<thumper> mwhudson: yep
<mwhudson> :/
<thumper> firstly rootsite from the canonical_url_data can be None
<thumper> moved the fix
<mwhudson> ah
<thumper> still have things that blowup
<mwhudson> just in the canonical url fix branch?
<thumper> yeah
<mwhudson> or in your one that adds layers everywhere
<mwhudson> :(
<thumper> I think the "correct" checking is causing other errors
<mwhudson> you mean exposing latent bugs?
<thumper> yeah
<thumper> haha, found one
<mwhudson> wheee
<mwhudson> so we have 404s from this today?
<thumper> not sure
<thumper> still checking
<thumper> lib/lp/translations/templates/product-translations.pt:47:                    <a tal:attributes="href target/fmt:url/+export"
<thumper> +export is only on the translations page
<thumper> however the canonical url of the product series is rootsite
<thumper> mainsite sorry
<thumper> so in the past this worked
<thumper> using the current request
<thumper> now it doesn't
<mwhudson> well also fmt:url/canonical_url used to prefer the link on the site of the current request
<wgrant> jelmer: Does your XB-* branch also publish non-XB-* fields?
<mwhudson> so it generated a good link before 3.0 even
<mwhudson> thumper: i guess the fix is target/fmt:url:translations/+export ?
 * thumper nods
 * mwhudson has surprising news: the blueprints code isn't very good
 * thumper is in shock
<wgrant> I do not understand why Blueprint has been left to stagnate.
<wgrant> Kill it off or fix it... don't leave it to drag LP down!
<mwhudson> we can't kill it without the ubuntu devs coming after us with pitchforks
<mwhudson> i imagine
<mwhudson> er
<mwhudson> make run_all is stabbing me in the face
<mwhudson> bzrlib.errors.NoWhoami: Unable to determine your name.
<mwhudson> i guess make -k run_all is a sort of workaround
<mwhudson> or not
<wgrant> Why would run_all be committing?
<thumper> wgrant: it calls run_codehosting
<thumper> wgrant: which makes real branches
<wgrant> Oh.
<wgrant> And makes fake branches.
<thumper> for those in the sample data
<wgrant> Rihgt.
<wgrant> That feature is annoying.
<wgrant> I want to remove it.
<thumper> heh
<thumper> it wouldn't bother me
<thumper> I never use the 'fake' sample data branches
<wgrant> "Let me just restart the appserver to test this recipe change..."
<wgrant> "WHERE did my branches go?"
<thumper> wgrant: fix it and I'll approve
<thumper> wgrant: we should clean codehosting on make clean
<thumper> not make run
<thumper> mwhudson: I might have fixed the bugs with 3 .pt edist
<thumper> edits
<thumper> 28 failing tests to check
<mwhudson> thumper: cool
<thumper> all for translations +imports and +export
<mwhudson> hooray for OAOO
<thumper> OAOO??
<mwhudson> (Once And Only Once)
<thumper> heh
<thumper> :(
<thumper> nope
<thumper> now links are being clicked and taken to translations sites
<thumper> or other sites on a different subdomain
<thumper> clucking bell
<wgrant> Wasn't there a bit of a proposal a while ago to remove the individual rootsites?
<thumper> I don't recall
<thumper> vague recollections
<thumper> but not likely
<thumper> down to 19 failures
<thumper> Link has site=, canonical_url has rootsite= :(
<thumper> yay consistency
<mwhudson> + layer + facet
<mwhudson> woo
 * thumper looks for the sarcasterisk
<wgrant> Is there any reason not to delete facets entirely?
<mwhudson> wgrant: no
<wgrant> Yay.
<wgrant> StevenK: The raw SQL in your branch makes me sad. But it looks good.
<thumper> wgrant: I have a branch that removes all facet definitions for LP code
<thumper> as in lp.code
<thumper> not all code in lp
<wgrant> Yay.
<wgrant> It still works correctly with tabs and breadcrumbs?
<thumper> seems to
 * thumper sighs
 * thumper cries quietly
<jelmer> wgrant: It just uses the control fields in the binary/source control fields, the XB- bit has already been stripped out at that point.
<wgrant> jelmer: Oh, true.
 * wgrant fails.
<thumper> 17 failures...
<thumper> lib/canonical/launchpad/pagetests/basics/notfound-traversals.txt takes a LONG time to run
<wgrant> But it's also deprecated.
<thumper> but no one will remove it
 * thumper taps fingers while the tests run....
<thumper> again
<jelmer> thumper: does CodeImport have any unittests?
<thumper> it should do
<thumper> lp.code.model.tests.test_codeimport
<thumper> would be my first guess
<thumper> and lp.code.stories.codeimport
<jelmer> ah, they're hidden under model/
<thumper> jelmer: we have model tests under model
<thumper> browser tests under browser
<thumper> interface tests under interface
<thumper> you know
<thumper> where they should be :)
<wgrant> I thought most apps just had lib/lp/APP/tests, and a couple had lib/lp/APP/browser/tests.
<jelmer> yeah, that's what confused me
<wgrant> I admit that model/tests probably makes more sense, though.
<thumper> wgrant: most apps do
<thumper> wgrant: but code blazes the "correct" way
<thumper> \o/
<wgrant> Code seems to be correct in a lot of ways :(
<thumper> wgrant: we still have lp.code.tests, but I'm trying to get around to shuffling them into the right place
<thumper> I've almost fixed all the translation story failures in my canonical_url branch fix
 * thumper crosses fingers
<james_w> I saw a presentation from a github dev the other day, and his overview of how to contribute equated to "grab trunk, hack, go to web site, fork, add a remote in git, then push to that remote"
<wgrant> But Git's better!!!
<james_w> so whereas I thought that server-side branch would be very useful for demos/beginners, they didn't take advantage of that, and had an even worse experience
<thumper> interesting
<thumper> james_w: I have a branch in progress (or feature if you will) that will mean you can register a project on LP, then go bzr push lp:project
<thumper> and it will make that branch the trunk
<wgrant> thumper: Who will own the branch?
<wgrant> Project owner?
<thumper> there have been suggestions that we should also create a project
<thumper> wgrant: the pusher owns the branch
<james_w> cool, but that's not my use case for demos
<wgrant> thumper: :(
<thumper> wgrant: it could be changed
<thumper> wgrant: it isn't hard
<thumper> wgrant: but it should be a separate fix (IMO)
<james_w> the maintainer of the project if one is set?
<james_w> (and it's a team that the pusher is part of)
<thumper> the push fails if the pusher doesn't have permission to set the link
<james_w> right
<thumper> ah FFS!!!
<thumper> ââ¹
<james_w> arghl OOPS-1693EA64
<james_w> thumper: I've just realised the workaround in buildrecipe for the bzr bug isn't going to cut it, as we execute bzr-builder with sudo
<mwhudson> james_w: luckily* the lp-oops application seems to have fallen over
<wgrant> james_w: Hm, it wasn't tested?
<james_w> wgrant: I guess not
<james_w> I was one of those that ACKed it and didn't spot it at the time
<mwhudson> ah, it's back now
<james_w> apparently not finding my oops though
<james_w> I think it's bug 518337 anyway
<_mup_> Bug #518337: timeout oopses on person page <timeout> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/518337>
<thumper> :(
<thumper> the counts that it adds up are taking over a second each
<thumper> down to 6 failing tests
<poolie> hello thumper
<poolie> thumper pitti says that https://code.edge.launchpad.net/~pitti/postgresql/common-dapper does not have an upgrade button for him
<poolie> even though it does seem to be a pack-0.92 branch
 * thumper looks
<thumper> poolie: we don't know what format it is in
<thumper> poolie: as it hasn't been touched forever
<thumper> since we don't know
<thumper> we don't show it
 * thumper sighs
<thumper> 4 failures...
<thumper> 3 failures
<thumper> 1 failure
<mwhudson> thumper: mind you, the lock/unlock trick will update the format now
<thumper> mwhudson: it will
<thumper> perhaps we should just twiddle everything we don't have formats for
<mwhudson> yeah, would be interesting
<mwhudson> i wonder how many weave branches there are...
<poolie> i think you should
<thumper> ec2 landing now
<mwhudson> thumper: congrats
<thumper> phew
 * thumper goes to have lunch
<StevenK> wgrant: It makes me sad too. It also makes me sad that I'm missing a table.
<wgrant> StevenK: Ah, PackageSetInclusion?
<StevenK> wgrant: Yup
<StevenK> I think I've figured out how to deal with that, too
<StevenK> Two extra queries :-(
<wgrant> Why so many?
<StevenK> I was going to run through packagesetinclusion where the child packageset is the current parent packageset and change it to the child, and then do the same for the child packagesets
<StevenK> wgrant: You don't agree, or I just confused you?
<wgrant> You confused me.
<wgrant> Then I decided to go and pester my useless project team.
<wgrant> So, can you reword?
<StevenK> wgrant: The problem is I have two many parent and child thing
<StevenK> *things
<StevenK> wgrant: I was going to run over packagesetinclusion where the child column is the parent packageset, and change it to the child packageset -- and then do the same for the parent column
<wgrant> StevenK: Can't you do that (and don't you need to do it?) in one query, just mapping parent and child to the new packagesets?
<StevenK> wgrant: I can't think of how to do in one query
<StevenK> wgrant: Suggestions are welcome
<wgrant> StevenK: I don't see how you could do it in two.
<wgrant> INSERT INTO packagesetinclusion (parent, child) SELECT <map old parent to new parent>, <map old child to new child> FROM packagesetinclusion blah blah blah
<wgrant> StevenK: How does my suggestion look?
<StevenK> wgrant: I don't get how to expand <map old parent to new parent>
 * mwhudson runs away, good weekend all
<thumper> wgrant: add some more details :)
<thumper> wgrant: like the rest of the SQL
<thumper> heh
 * thumper wants a beer
<wgrant> thumper: I would have if I wasn't really busy. I will in half an hour :)
<wgrant> Ah, this is all within a parent-specific iteration. I seee.
<StevenK> wgrant: So now you agree with me?
 * StevenK heads to the doctors
<wgrant> StevenK: Well, I don't think that loop should be there.
<wgrant> StevenK: But regardless, surely just running through all the child packagesets is sufficient. You don't need to run through the parents too -- that would get everything twice.
<jtv> jml: just having a look (belated, I know) at your ReadyToCode checklist.  Is that really what "ready to code" means?  Doesn't some notion of approach and feasibility come in before that stage?
<poolie> noodles775: hi, is the context of your mail about ppas "if this was done it would answer martin's question"?
<poolie> hi jml, jtv
<jtv> hi poolie
<noodles775> poolie: Yeah, I saw you asking about ppa download logs, but didn't know the context of your irc chat... just sent the email in case it answers your question.
<poolie> i want to find out how much people use old distroseries from our ppas
<wgrant> It's implemented, but waiting on LOSAs :(
<noodles775> Yes, but we haven't yet been able to process the backlog of ppa logs (due to the memory issue). The email I sent will hopefully enable us to start processing them, but as wgrant says.
<nigelb> wgrant: heh, poke spm in the eye?
 * wgrant sees no such email.
 * noodles775 forwards it to wgrant.
<wgrant> noodles775: But the memory issue should be fixed now, right?
<noodles775> wgrant: apparently not. Well, when it was run, we still hit the issue (but were a bit optimistic with our default number of lines)... the email has more detail.
<wgrant> noodles775: Ah, I see. Thanks.
<wgrant> I wasn't aware that work was actually ongoing to get it running.
<poolie> and this will answer that question, when it's done?
<wgrant> You will be able to grab download counts per country and date for each .deb in a PPA.
<wgrant> Only through the API for now, though.
<poolie> and then summarize it myself
<poolie> anyhow, that would be very nice
<poolie> i think i'd actually be more interested in knowing how many IPs polled the ppa
<poolie> but either will do
 * poolie thinks the less launchpad makes me use my mail client, the better
<wgrant> I considered that. But there wasn't existing code to do that sort of thing, and it's less obvious how it should be counted.
<poolie> well, thanks very much for doing it
<wgrant> It'll be good once it can actually be turned on safely.
<adeuring> good morning
<mrevell> Hello
<jtv> hi mrevell!
<StevenK> wgrant: But that doesn't cover the case of parent packageset being copied too
<wgrant> StevenK: Hm, does PackagesetInclusion work across distroseries boundaries?
<wgrant> I guess so :(
<StevenK> wgrant: So you finally agree with me, or you have a better idea?
<wgrant> I don't agree that you need two queries. However, an inter-series PsI brings about a situation where the behaviour is undefined.
<wgrant> What should be done in that case?
<StevenK> al-maisan was suggesting looping over at the end
<wgrant> To do what?
<wgrant> I don't actually know what the behaviour in this case should be.
<wgrant> It's certainly not obvious.
<StevenK> To copy the records from packagesetinclusion
<wgrant> Yes, but what does that mean? What if I have a PsI with its parent in karmic, and child in lucid. I am copying lucid into maverick.
<wgrant> How do I copy that?
<StevenK> The child packageset changes to maverick and the parent stays as karmic?
<wgrant> I guess.
<StevenK> This is a little sub-optimal
<wgrant> Now, what if it's the other way around?
<StevenK> parent moves and child doesn't
<wgrant> In one of the directions, creating a new series is going to magically add more packages into an older series' packageset.
<StevenK> Yes, adding more children is dangerous
<al-maisan> Isn't the idea that each series has its own package sets i.e. changes to the latter in a child series would not affect the parent series?
<wgrant> al-maisan: Yes.
<wgrant> But there's nothing in PackagesetInclusion to require that.
<wgrant> And while it will probably never happen, I realllly don't want to leave undesirable behaviour in something as critical as i-f-p.
<al-maisan> I did not quite get the "creating a new series is going to magically add more packages into an older series' packageset" statement..?
<wgrant> al-maisan: If I have a PackagesetInclusion with a parent in karmic, and a child in lucid, creating maverick from lucid will mean that karmic inherits from maverick too.
<al-maisan> wgrant: you mean: it is possible to associate package sets across series in PackagesetInclusion i.e. there's no db constraint precluding that and you see that as a risk?
<wgrant> al-maisan: Pretty much.
<wgrant> I'm not sure how much of a risk it is.
<wgrant> But it needs to be examines.
<StevenK> Well, we could run a query on staging?
<wgrant> There shouldn't be any data like that now.
<wgrant> But some could spring up in the future. And then i-f-p does something very confusing and dangerous without telling anybody.
<wgrant> We might just ignore the risk. But we need to explicitly decide that.
<StevenK> Well, currently, i-f-p does nothing of the sort?
<wgrant> Once it starts copying PsI it will.
<al-maisan> Packageset._addDirectSuccessors() (in lib/lp/soyuz/model/packageset.py, line 114) could be refined to prevent that from happening
<wgrant> al-maisan: Having that restriction outside the DB makes me sad, but I guess it's the best that can be done.
<al-maisan> .. and a db constraint on INSERT/BEFORE into PackagesetInclusion could be easily formulated
<al-maisan> hmm .. there is already an INSERT/AFTER trigger on packagesetinclusion : packagesetinclusion_inserted_trig
 * al-maisan never tried having a trigger before *and* after an INSERT on the same table
<wgrant> I guess that's there to update flatpackagesetinclusion.
<wgrant> Yep.
<al-maisan> I'll do a quick test when I have some spare time and let you know
<al-maisan> maybe stub would know off the top of his head whether it's ok to have a trigger before *and* after an INSERT on the same table..?
<StevenK> I don't see why that would be an issue, personally
<al-maisan> if that works we could add the before insert trigger on packagesetinclusion and veto associations of package sets across distro series
<wgrant> That seems to make sense.
<wgrant> And it makes me less wary about breaking Ubuntu...
<al-maisan> wgrant: maybe you could open a bug and I can look into it in the next couple of days..?
<wgrant> al-maisan: If you could. If not, I can look at fixing it at some point.
 * wgrant files.
<al-maisan> who ever gets to it first :)
<wgrant> Bug #620989
<_mup_> Bug #620989: Package sets can include sets in other series <Soyuz:New> <https://launchpad.net/bugs/620989>
<al-maisan> thanks
<al-maisan> for filing :)
<wgrant> StevenK: Did hacking Storm's Insert to support 'INSERT FROM ... SELECT' prove difficult? I'm just wondering if you can warn me before I get lured into a deceptively simple trap.
<lifeless> mthaddon: hi; just wanted to say I'm really glad to hear we can move forward on the additional QA environment soonish.
<lifeless> wgrant: looked entirely straight forward to me
<mthaddon> lifeless: yep
<lifeless> mthaddon: if there is anything I can do to make it simpler/unblock things, please shout!
<mthaddon> will do
<lifeless> mthaddon: separately
<lifeless> do you know when we're likely to get edge moving again :)
<lifeless> as in, is it waiting a dev to change something, or losa cycles, or cron?
<mthaddon> lifeless: hmm, it was something spm was working on, but he was ill today - I'll get out and push
<lifeless> thanks
<StevenK> wgrant: I didn't hack on it, lifeless said there ways around it, but my lack of knowledge about Storm defeated me
<wgrant> Ah, OK.
<lifeless> is this evil ?
<lifeless> milestones = search_results._store.find(
<lifeless> the _store I mean
<lifeless> why would that be needed?
<wgrant> It's wrong.
<lifeless> gmb: ^ yo, its in bugtask, I blame you.
<lifeless> wgrant: what should it be ?
<wgrant> Store.of(search_results) is the right way to spell that.
<lifeless> its blowing up because DRS doesn't have that attribute
<wgrant> But do you really want to do that rather than ISlaveStore(something)?
<lifeless> I don't know
<lifeless> I'm guessing it sometimes runs in posts
<lifeless> and rather than explicit its trying to be magical.
 * gmb has no idea why that's there.
<lifeless> magical magical goodness. 'goodness'
 * gmb bzr blames...
<lifeless> I have only 46 failing tests with this bugtaskset overhaul
<gmb> allenap, J'accuse!
<lifeless> well
<lifeless> 46 in my current reduce() phase
<lifeless> I may have broken a tonne of other shit
<allenap> gmb: What? :)
<gmb> allenap, Using _store. Apparently, this is bad.
<wgrant> _. 'nuff said.
<allenap> gmb: Yeah, I agree. Did I do it?
<lifeless> es
<lifeless> and my patch to cut hundreds of queries out of the landscape milestone 'later' page loads is blowing up in there
<lifeless> because DecoratedResultSet doth not have _store
<gmb> allenap, Apparently. TBH, I just wanted lifeless to blame someone else so I could get back to my JS horrorshow uninterrupted.
<lifeless> whats the magic env variable again, for SQL output
<lifeless> LP_DEBUG_STORM or something?
<allenap> lifeless: I guess Store.of(self) should work fine there.
<wgrant> lifeless: LP_DEBUG_SQL and LP_DEBUG_SQL_EXTA, IIRC.
<allenap> lifeless: LP_DEBUG_SQL and LP_DEBUG_SQL_EXTRA
<wgrant> *EXTRA
<lifeless> allenap: its a BugTaskSet
<allenap> lifeless: Oh balls.
<lifeless> allenap: I'm trying Store.of(self)
<lifeless> allenap: well, this is a reason not to have Sets as currently conceived
<lifeless> and instead collections which are more query builders yada yada yada
<lifeless> it would be nice to have an explicit mapper structure
<jml> jtv, before which stage?
<jtv> jml otp
<allenap> lifeless: One option is to modify the c.l.webapp.adapter.get_store adapter so that you can use IStore(search_results).
<lifeless> allenap: well, if Store.of(DRS) works, then I'll use that.
<allenap> lifeless: Ah, yes, that would be better. I just assumed it wouldn't :)
<lifeless> allenap: is there any reason this can't ?
<lifeless> I mean, I assume it will be pain and heartache
<lifeless> the other question
<lifeless> is that there is a query
<lifeless> bringing in milestones already
<jml> in any case, hello
<lifeless> it seems nuts to me to be querying bugtasks, extracting IDs and throwing away th eobjects and then requerying for milestones
<lifeless> why not use the other query as an alias
<lifeless> and just do a single query
<lifeless> jml: ho hai
<allenap> lifeless: IResultSet is delegated to the real result set, and that doesn't include __storm_object_info__ which is what Store.of() relies upon (via get_obj_info).
<lifeless> Decorated is what you mean, and yes, Isee that
<lifeless> (I'm in a big hariry branch stormifying bugtaskset
<lifeless> not entirely by choice
<lifeless> anyhow, I'm going to take the 'do this later approach
<lifeless> but I stand by my 'two queries here is nuts' comment
 * lifeless encommenates
<allenap> lifeless: I agree with the "two queries here is nuts" comment. I can't remember why it happened that way. If it was me, I may have not understood or been able to get aliases to work, and just got this working for expediency.
<lifeless> sqlobject
<allenap> Possibly.
<lifeless> hmm
<lifeless> may be closing in on 'working'. zomg.
<lifeless> jml: got a few ?
<jml> lifeless, sure.
<wgrant> lifeless: How much of the milestone index is SQL time?
<lifeless> jml: you cut out
<lifeless> wgrant: all of it
<lifeless> wgrant: (more or lesss)
<lifeless> wgrant: one query per private bug
<wgrant> lifeless: Oh, I heard there was lots of Python.
<wgrant> And that's why memcached was used.
<lifeless> wgrant: see my perf tuesday mail this week
<lifeless> yeah, not a memcache issue
<lifeless> jml: https://edge.launchpad.net/~bobthebuilder
<lifeless> gnight
<jelmer> 'night lifeless
<deryck> Morning, everyone.
<salgado> bigjools, I assume it's ok to mark bug 510892 and bug 510894 as qa-untestable, right?
<_mup_> Bug #510892: Upload policies are instantiated unnecessarily <cleanup> <qa-needstesting> <tech-debt> <wellington> <Soyuz:Fix Committed by salgado> <https://launchpad.net/bugs/510892>
<_mup_> Bug #510894: Upload policy names are duplicated with the actual builds <cleanup> <qa-needstesting> <tech-debt> <wellington> <Soyuz:Fix Committed by salgado> <https://launchpad.net/bugs/510894>
<bigjools> salgado: I think so, yes.
<jml> how would I find out how many different pages we have?
<jml> "find . -name '*.pt' | wc -l" gives me a rough approximation.
<jml> sinzui, perhaps you'd know?
<noodles775> jml: the google analytics don't do that? (it's been a while since I've used it)
<noodles775> (assuming we've got the ga js code on each page)
<jml> noodles775, by number of pages, I mean "bug-index" would count as one
<jml> noodles775, would GA help with that?
<noodles775> jml: I'm not sure, but thought it would (given that each page has the ga js snippet)...
<wgrant> Does GA know the page ID, or just the URL?
<jml> nfi.
<noodles775> wgrant, jml: ah, sorry - I see what you mean now.
 * jml is looking for the GA details.
<jml> getting a list of page ids would be a good start.
<sinzui> jml google analytics is the best answers
<jml> what makes up page ids?
<wgrant> Class:+page
<wgrant> Or you mean what calculates them?
<jml> well, what I *actually* mean is "why isn't there a script I can run to get me a list"
<jml> but yeah, what calculates them.
<sinzui> jml: flacoste wrote it. It is object + view and I recall it does not work if we registered an object with a template
<wgrant> sinzui: See constructPageID in c.l.w.publication
<wgrant> Er.
<wgrant> jml: ^^
<jml> wgrant, thanks.
<wgrant> But not useful for your purposes :(
<jml> useful enough.
<jml> Now that I know how it's calculated, I bet I could extract the info I need from ZCA
<wgrant> Well, views are just adapters, so yeah.
<jml> although I guess there's no way of distinguishing views that are pages from views that are not.
<wgrant> Well, lots that aren't will have 'portlet' in the view name. And most of the rest that aren't will have '-macros' in the template name.
<jml> heuristics yay
<wgrant> Yes :(
<jml> ok. shelving this problem for now. perhaps after lunch, code review, meetings, email and my weekly organizational cleanup.
<salgado> bigjools, I've got a branch which changes a bunch of tests that were relying on mixed uploads to not rely on that anymore, as those don't happen in practice.  would you like to review it?
<wgrant> I need to refactor stuff so we can remove most of those test uploads.
<deryck> sinzui, hi.  I'm having a poke at your email and branch about heat and bug counts now.
<sinzui> deryck, thanks
<deryck> sinzui, so if I'm playing locally, say at https://launchpad.dev/ubuntu/hoary/+needs-packaging ...
<deryck> ah, crap.  he has gone.
<bigjools> salgado: I don't need to see that, you can use the OCR if you don't mind
<salgado> bigjools, sure, that's fine with me
<bigjools> cheers
* mthaddon changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 1 of 10.09 | PQM is OPEN | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | mailman down approx 13:30 - 14:30 UTC for lucid upgrade
<deryck> sinzui, so I see two things that I believe will fix your need-packing page counts....
<deryck> sinzui, one is to move your calls to task.target.recalculateBugHeatCache() inside updateHeat, not setHeat....
<deryck> sinzui, and the other is that you need to filter out dupes as well as closed bugs.
<sinzui> ahh@
<sinzui> deryck, thank you very much
<sinzui> I had not considered dupes at all
<deryck> sinzui, no worries.  I'll reply on list, too, just for the sake of anyone else playing along.
* mthaddon changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 1 of 10.09 | PQM is OPEN | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews
<jml> I'm stopping.
<jml> Have a good weekend everyone.
<lifeless> ciao
<lifeless> whats the batch size trick again
<lifeless> to turn batching off
<mtaylor> merging accounts seems to lose karma ...
<lifeless> how so
<EdwinGrubbs> lifeless: what is LBYL?
<lifeless> look before you leap
<lifeless> aka EAFP
<nigelb> European Association of Fish Pathologists?
<lifeless> eaier to ask forgiveness than permission
<nigelb> Aha
<nigelb> lifeless: wait a minute, didn't I see you before I went to bed last night?
<nigelb> long day/
<lifeless> theres a general pattern in python, and it applies to performance as well, where asking-in-advance can be slow (unless fancy caching is employed) compared to just doing the thing (but once and only once)
<lifeless> nigelb: its 0935 here
<nigelb> lifeless: ok, my mistake.  3 am.  should sleep.
<lifeless> :P
#launchpad-dev 2010-08-21
<lifeless> browser code isn't meant to be doing SQL, is it ?
<jelmer> lifeless: No
<jelmer> lifeless: I mean, no it's not supposed to be doing SQL.
<lifeless> browser/bugtask.py line 3107 may scare folk
<jelmer> lifeless: I think the import fascist may actually error out if you try to import database stuff
<lifeless> removeSecurityProxy ftw
<lifeless> bbiab
#launchpad-dev 2010-08-22
<lifeless> fg
<lifeless> bah, typo ;P
<lifeless> 5 fails.
<lifeless> the countdown continues
<lifeless> wgrant: what do you think of my storm/sqlobject suggestion ?
<lifeless> wgrant: its taken considerable time tracking down essentially all cases of this, for *one* function that returned a SQLObject query becoming storm.
<wgrant> lifeless: Storm.__nonzero__ should certainly fail.
<wgrant> Not sure about the warning, though.
<lifeless> well
<lifeless> making it fail always would be a bit harsh
<wgrant> True.
<wgrant> s/Storm/ResultSet/ in my above, oops.
<wgrant> But true.
<lifeless> the warning is for catching problems early - I'm thinking one could do a run again ec2 and generate a list of callsites
<lifeless> \o/
<lifeless> fingers crossed, the damn branch is done
<wgrant> Which?
<wgrant> The search one?
<wgrant> BugTaskSet.search?
<lifeless> yeah
<lifeless> the one that became
<lifeless> 'convert a workhorse function to storm, somehow'
<wgrant> Hah.
<lifeless> buildQuery + search is not a simple thing :<
<lifeless> hah
<lifeless> 988 __metaclass__ = type
<lifeless> just under the 1000 line limit
<wgrant> What 1000 line limit?
<wgrant> You mean the 800 line review limit?
<lifeless> ah heh
<lifeless> used to be 1K
<lifeless> I think
<lifeless> anyhow
<lifeless> I'll beg :P
<lifeless> hmm
<lifeless> can't search for private bugs ?
<lifeless> gnight
<lifeless> anyone seen this before ?
<lifeless> WARNING:root:Memcache set failed for pt:testrunner:lp/bugs/templates/bugtarget-portlet-latestbugs.pt,10721:243652:1,0:1,http_//127.0.0.1?IXiDEgHZPmgHvZKrJmNvh
<james_w> could someone clear out old things from lp-source-dependencies?
<maxb> lp-source-dependencies?
<maxb> oh, the download-cache project :-)
<maxb> And yes, please could someone do that? :-)
<james_w> I don't think we need a copy of the tarball of the bzr 1.16 release on everyone's machines, nor a dozen releases of lazr.restful.
<maxb> nor python 2.4 eggs :-)
<maxb> or 10 versions of storm :-)
<lifeless> bombs away
<jelmer> 'morning lifeless, james_w, maxb
<lifeless> hi jelmer
<lifeless> jelmer: you should ask for a db review from stub too
<jelmer> lifeless: ok
<lifeless> I have done it in this instance
<jelmer> lifeless: do I need two reviews or just because either of you can review?
<lifeless> but he is authoritative
<lifeless> you need two
<lifeless> one to land, once you have a db patch id
<lifeless> this is documented on the wiki
<mwhudson> morning
<jelmer> hellow
<thumper> morning
<mwhudson> thumper: congrats on that canonical_url fix finally landing
<thumper> mwhudson: thanks
<jml> hello antipodeans
<mwhudson> jml: hello ex-antipodean
<lifeless> hawwo
<thumper> jml: are you skypable?
<jml> thumper, sure. gimme a few minutes first though.
<thumper> jml: sure, ping me when you are ready, I'm on a different v.desktop
<jml> thumper, ping
<weather15> Hello Everyone
<weather15> I am trying to install Launchpad on my own server and I seem to be getting this error: Logging bzr into Launchpad (it's okay if this errors)... Making local branch of Launchpad trunk, this may take a while... Permission denied (publickey). bzr: ERROR: Connection closed: Unexpected end of message. Please check connectivity and permissions, and report a bug if problems persist. ERROR: Unable to create local copy of Rocketfuel tru
<weather15> seems to be an SSH key problem
<weather15> Just re-created mu ssh key and then updated it on the Launch Pad site and everything is working now
<thumper> I'd like to welcome wallyworld to the LP dev team
<lifeless> wallyworld ?
<wgrant> Yay, more Australians :)
<thumper> lifeless: Ian Booth :)
<lifeless> welcome
<lifeless> wgrant: you had me up to australians :P
<weather15> It seems when trying to install Launch Pad I get this error: bazaar.launchpad.net/~launchpad-pqm/shipit/trunk/
<weather15> I browsed to the url and I get Unauthorized
<weather15> If this an error I should ignore or is it more serious?
<weather15> *Is
<wgrant> You can ignore it, and everything will still work fine. ShipIt isn't accessible to community devs
<weather15> okay can I ask one more question?
<weather15> I ran cd devel then  ./utilities/launchpad-database-setup $USER and then make schema for some reason the command seems to hang
<weather15> Heres the output: utilities/shhh.py PYTHONPATH= python bootstrap.py\                 --setup-source=ez_setup.py \                 --download-base=download-cache/dist --eggs=eggs rm -f /home/weather15/launchpad/lp-branches/devel/bin/py utilities/shhh.py PYTHONPATH= ./bin/buildout \                 configuration:instance_name=development -c buildout.cfg
<weather15> It is still running yet how ever
<wgrant> How long has it been running?
<weather15> Should I stop the script? Or ..
<weather15> or is there a problem because i ran the command in the dev directory?
<weather15> The wiki says to
<weather15> ?? ^
<wgrant> weather15: How long has it been running?
<weather15> 10 minutes
<wgrant> Is it doing anything?
<weather15> Nope
<weather15> Just gave the output I posted above and then nothing else it's just hanging
<weather15> New Output
<weather15> Unknown entry URL:                     translation_file
<weather15> and a few more like that
<wgrant> It's working.
<weather15> any idea as to what ^ this means?
<wgrant> It's building.
<wgrant> Give it a couple more minutes.
<weather15> Okay I will
<weather15> Never know what I might come across though
<weather15> Warning: No permissions specified for [u'public.openidnonce', u'public.openidauthorization']
<wgrant> that's normal.
<weather15> the script has completed
<weather15> I just sent my system down for reboot before I start to run it
<weather15> Linux Core was updated
<weather15> Then we wil see what happens
<weather15> Okay once you start it it tells you to access that url from a local web browser
<weather15> I am runnign Ubuntu Server so there is no web browser
<weather15> *running
<weather15> Error Starting: [Errno 2] No such file or directory: '/var/tmp/mailman/data/master-qrunner.pid' Is qrunner even running?
<weather15> wgrant any idea as to what to do about this ^?
<wgrant> weather15: Ignore it. It's not a problem.
<weather15> Okay Great Thanks
<weather15> What should I do about the Web Browser?
<wgrant> https://dev.launchpad.net/Running/RemoteAccess
<weather15> Okay Great Thanks for all of your assistance.
#launchpad-dev 2011-08-15
<lifeless> yay @ 3KB/s from LP
<lifeless> (not LP's fault - the weather is messing up my internets)
<wgrant> The cold should be improving conductivity!
<nigelb> lifeless: Morning
<nigelb> lifeless: It was just one page
<nigelb> wgrant: Everyone must be home and using the internet in the cold ;)
<lifeless> http://www.stuff.co.nz/national/5434460/NZ-set-for-perfect-snowstorm
<nigelb> Also, it was a db heavy page, so now I think it was probably okay.
<StevenK> Sounds right. You should listen to lifeless talk about his Internets at 5pm
<lifeless> *hates*
<nigelb> haha
<nigelb> oh. Today's a Monday and its a holiday! <3
<lifeless> grah
<lifeless> so.umask
<lifeless> ...
<lifeless> and then restores it
<lifeless> no finally.
<lifeless> no cleanup.
<lifeless> -> how to mess up an entire test run if one test fails.
<lifeless> poolie: could you spare a few minutes to teddy bear a design with me ?
<lifeless> nigelb: better pics - http://www.stuff.co.nz/national/5443420/Major-weather-disruption-around-NZ
<nigelb> lifeless: it looks beautiful
<nigelb> of course, I'm saying this while bathed in sunlight ;)
<lifeless> nigelb: better pics - http://www.stuff.co.nz/national/5443420/Major-weather-disruption-around-NZ
<poolie> nice
<lifeless> time to move the python-oops code to python-oops_datedir_repo
<lifeless> StevenK: any chance I can get a couple of reviews from you?
<lifeless> https://code.launchpad.net/~lifeless/python-oops/extraction/+merge/71507 and https://code.launchpad.net/~lifeless/launchpad/useoops/+merge/71506
<lifeless> StevenK: mainly deletes again
<StevenK> lifeless: Having desktop issues
<StevenK> steven@liquified:~% ps axw | grep -c "<defunct>"
<StevenK> 153
<lifeless> StevenK: ugh
<lifeless> StevenK: \o/
<StevenK> ^ That isn't helping
<lifeless> StevenK: sure its not a lp test run parent ?
<StevenK> No, they're all gnome stuff
<lifeless> heh
<lifeless> https://launchpad.net/python-launchpad-filesystem
<StevenK> # XXX AaronBentley 2009-11-26 bug=488950: There should be separate storage for oops messages.
<StevenK> lifeless: Haven't you dealt with that?
<lifeless> StevenK: no
<lifeless> StevenK: by the end of the arc I will have
<lifeless> but changing that means fixing the oops-tools bug about actually using the common extracted code.
<StevenK> lifeless: Both approved
<lifeless> thank you!
 * StevenK tries to swap back the state he had before the reviews, fails
<lifeless> :(
<lifeless> sorry!
<StevenK>   File "/home/steven/launchpad/lp-branches/link-branch-to-private-bug/lib/lp/code/model/branch.py", line 181, in setPrivate
<StevenK>     raise BranchCannotBePrivate()
<StevenK> :-(
<lifeless> probably nee da privacy policy
<lifeless> or non-junk or non-distro branch
<StevenK> I've been trying to work that out
<StevenK> The code isn't being very helpful
<StevenK> lifeless: So I have a product branch, and it still raises the exception. But I don't want the branch to be private by default -- I'm trying to go for what we do with LP itself.
<lifeless> are you making it private as an admin ?
<StevenK> No, I'm not.
<StevenK> I'm wondering if I should
<lifeless> I suspect you need to.,
<lifeless> AFAIK we have no 'user can opt in' story that is polished :)
<StevenK> This is in model code, so I probably need to removeSecurityProxy()
<StevenK> Which makes me sad.
 * StevenK grumbles
<StevenK> IBugBranchSet.getBranchesWithVisibleBugs iterates over a collection of branches it is passed, and doesn't deal with unauthorized errors
<StevenK> lifeless: Can haz review?
<lifeless> if you give me a url
<lifeless> StevenK: I see it
<lifeless> uhm
<StevenK> lifeless: Um?
<lifeless> uhm
<lifeless> reviewed; twice :)
<lifeless> I have to go cook v. soon but I can discuss later, or tomorrow
<StevenK> lifeless: Public branches can become private -- we use this in LP occasionally
<lifeless> admin only functionality
<StevenK> Right
<lifeless> with this change as it stands, I can make anyone's branch private
<StevenK> I will discuss it with Curtis tomorrow
<lifeless> and without a privacy policy, they can't even undo that IIRC
<lifeless> I've commented in the bug
<lifeless> you're going to love the next review :P
<StevenK> Find someone else? Like Henning? :-)
<lifeless> sure :)
<lifeless> I've deleted the guts of python-oops, moved it to python-oops-datedir-repo, and added a new new core :)
<StevenK> datedir-repo?
<StevenK> Why?
<lifeless> because thats what the code really is all about
<lifeless> the structure we need is getting clearer the more I pull things apart
<lifeless> the lp oops system currently uses an oops repository based around directory named by the date
<lifeless> anyhow - for whomever feels like it - https://code.launchpad.net/~lifeless/python-oops/extraction/+merge/71510 +248 - 847
<stub> lifeless: not sure if it isn't core. Can you imagine an installation that doesn't use the directory storage? I thought it would be required as a backup for when more advanced data stores fail.
<stub> lifeless: I'm confused in the first lines of the readme - factory = Config(). Is it a misnamed method or a misnamed variable?
<stub> I'm used to factories having __call__ methods rather than create, but I suspect that is just personal style. I guess create is clearer as the factory needs to be configured before use.
<stub> Not really a factory is it with .publish - more an oops_handler
<stub> lifeless: The report is created by the oops_handler. Any reason to prefer oops_handler.publish(report) over report.publish() ?
<lifeless> stub: reports are just dicts
<lifeless> stub: see the README in the lp:python-oops
<lifeless> stub: thats deliberate for interop
<lifeless> stub: yes, I can imagine an install that doesn't use the directory storage
<lifeless> stub: e.g. local mq on the box, let that take care of downtime handling
<lifeless> stub: or even queue in process: by the time you can restart the process, you've gotten network back :)
<stub> I was wondering if the core library would use it as a fallback for exceptions happening in the pluggable storage.
<lifeless> stub: I think its best not to, so that the core really is lean - e.g. for inclusion on the Ubuntu CD
<lifeless> stub: where they'd want to use the apport *style* of disk store (one crash per program per user stored at max)
<stub> So exception handler would be logging.log.exception('Unable to store OOPS')
<lifeless> stub: perhaps; I think there is room for discussion around that. I can well imagine getting bug reports either way.
<lifeless> stub: for now, my focus is on a reasonable structure for the project(s), and acheiving enough extraction that gpgverify can import and use the library to do oops reporting
<lifeless> unblocking the SOA project
<lifeless> stub: if that makes sense
<lifeless> stub: another way to do fallbacks would be a publisher clients can use that: looks for an id, if not present then log via logging.error that something is wrong
<danilos> lifeless, hi, what's the status of our collided changes (re 728220): I could confirm OOPSes produced tonight have the data properly sanitized, so I just marked the bug qa-ok
<lifeless> stub: I don't think there is a single right policy for all users
<stub> Sure. I do think the OOPS system should never hard fail, so will need some sort of global exception handler and ability to emit meeps when your publication messes up or your dict isn't serializable.
<lifeless> danilos: I resolved it all
<danilos> lifeless, excellent, thanks
<lifeless> stub: agree on never hard failing; want to think more about what/how to do for that - at the moment we do hard fail if things blow up, so its not a regression in any sense
<lifeless> danilos: I moved it into the *make* rather than the *write*
<stub> lifeless: We currently put that burden on the call sites in Launchpad.
<lifeless> stub: overnight scan-branches blew up, its oops writer failed (directory permissions), so it propogated an exception to the root :)
<danilos> lifeless, I assume make happens before the write, which is even better
<stub> (appservers get it for free with publication, and scripts don't bother and would just explode on disk full)
<lifeless> stub: yeah
<lifeless> stub: so there is room for us to think about an interface which will let scripts to better and not burden appservers.
<lifeless> danilos: yes, exactly.
<lifeless> danilos: you can poke at my merge - rev 13684 - if you're interested.
<stub> Yup. try: [code calling non-core] except Exception: log.exception('OOPception') would be fine, assuming all the logging system callbacks are also bulletproof :)
<danilos> lifeless, thanks
<lifeless> stub: perhaps, though for an appserver that would double-log
<lifeless> stub: so I think this needs a bit more consideraiton
<stub> When did we start naming_modules_with_underscore?
<lifeless> stub: well, its not lp core, so its more pep8 than anything else
<lifeless> which says 'when it aids clarity'
<adeuring> good morning
<lifeless> stub: so was that by way of review or just discussion ?
<stub> lifeless: r=stub with comments
<lifeless> stub: thanks!
<StevenK> lifeless: O hai -- you remember the denorm branch you looked at on Friday? That should be fine to just lp-land, right?
<lifeless> StevenK: hahahahahahahahahaahahah
<StevenK> stub: O hai. So you think we should add indexes for BPPH.BPN and SPPH.SPN before we do the population?
<nigelb> lifeless: that was brilliant reply.
<nigelb> Need to have a quotes db.
<nigelb> one of these days I'll host it myself.
<StevenK> nigelb: SILENCE
<danilos> henninge, hi, will you be reviewing today? if so, I've got a branch up at https://code.launchpad.net/~danilo/launchpad/bug-690568/+merge/71394 ;)
* henninge changed the topic of #launchpad-dev to: Das Thema fÃ¼r #launchpad-dev ist: https://dev.launchpad.net/ | On call reviewer: henninge | Critical bugs: 238 - 0:[#######=]:256
<henninge> danilos: I will. ;-)
<henninge> danilos: can you fix the lint, please?
<danilos> henninge, sure, I didn't want it to pollute the diff
<henninge> danilos: that's something we'll have to live with if we ever want to get any transitions done.
<danilos> henninge, I usually do it post-review though
<henninge> oh, that is a good idea, too.
<henninge> danilos: cool
<StevenK> henninge: O hai, are you free?
<henninge> StevenK: not right now but feel free to queue ;-)
<StevenK> henninge: https://code.launchpad.net/~stevenk/launchpad/use-shutdown-ec2/+merge/71522 when you have a tick. Nice small one.
<stub> StevenK: If you don't have an index in place, the population will go really slow (it will take ages to calculate WHERE spn IS NULL)
<StevenK> stub: Right. I'm doing development of this in a pipe, I will shift a pipe into the middle for indinces
<henninge> danilos: r=me
<henninge> StevenK: Looking now
<danilos> henninge, thanks
<StevenK> henninge: Thanks!
<henninge> StevenK: did you change from shutdown -h to shutdown -P on purpose?
<henninge> sounds better to me , just wondering if there may have been a reason not to use -P before.
<StevenK> henninge: Since the other shutdown call used -P
<henninge> StevenK: Ah! (that's not in the diff obviously)
<henninge> StevenK: you mention a bug report but there is no bug linked
<StevenK> henninge: No, but the bug is mentioned in the code I removed
<henninge> duh
<henninge> StevenK: I assume you have already tried this out?
<StevenK> I have not
<StevenK> henninge: It was going to be tested with the landing
<henninge> ah, of course ;-)
<henninge> StevenK: r=me
<StevenK> henninge: Thanks!
<wgrant> gmb: aaaaa
<wgrant> gmb: Not sure that's a good one to pick.
<gmb> wgrant: Any reason other than "That bug will eat your face and you will die?"
<wgrant> given it's a) difficult, b) DB changes, c) we
<wgrant> .. might delete nominations later this year
<wgrant> d) difficult
<wgrant> e) face eating
<nigelb> f) kitten will die?
<stub> nom nom nom
<wgrant> There are pretty unfortunate complexities around SourcePackage tasks that this would eliminate.
<wgrant> But...
<wgrant> Ubuntu and lifeless want a rework of how series tasks work.
<gmb> Ah. Hmm.
<wgrant> So I'm not sure it's worth putting a huge amount of time into fixing my second oldest Launchpad bug.
<nigelb> wgrant: I have up trying to read Zope 3. Much eeasier to read LP code sadly.
<gmb> wgrant: Is there any kind of concrete schedule for said reworking? I'm more than happy to punt this if so, but I want it off the Critical list because I keep seeing it and fooling myself into thinking I can fix it.
<wgrant> At least not until we know what's happening.
<nigelb> *gave
<wgrant> gmb: You could just fix the OOPS.
<gmb> wgrant: Ah, good idea.
<wgrant> It only affects a few bugs from before 2007.
<henninge> Do changes to security.cfg need to be submitted to db-devel?
<lifeless> henninge: no
<henninge> lifeless: thanks
<lifeless> dev.launchpad.net/PolicyAndProcess/DatabaseSchemaChangeProcess 9r something like that
<wgrant> henninge: New permissions are applied before every nodowntime rollout, but revocations are only performed during downtime.
<wgrant> Regardless, should be landed on devel.
<henninge> ok
<lifeless> gmb: which bug?
<gmb> lifeless: https://bugs.launchpad.net/launchpad/+bug/110195
<_mup_> Bug #110195: Nominating a bug for a distro release affects all package tasks for that distro <lp-bugs> <motu> <oops> <Launchpad itself:In Progress by gmb> < https://launchpad.net/bugs/110195 >
<lifeless> gmb: I would fix it by making the thing that fails fail cleanly, vs oopsing
<lifeless> gmb: that will cover the oops bit, because its not something we need to investigate each time it happens
<nigelb> grr. posting to +check-links with extra information creates 500 error :|
<lifeless> gmb: you could fix it fully if you want, but as wgrant says, we probably stand to gain a lot by massively simplifying things - no non-series tasks, and nomination a task status
<gmb> lifeless: Yes, I agree; I'll make the OOPS not an OOPS and leave it at that for now.
<nigelb> guys, if I have a bug number, how do I lookup the validity of the bug?
<nigelb> err, inside LP that is.
<nigelb> Is there something similar to lp.code.interfaces.branchlookup.IBranchLookup?
<nigelb> I wonder if I bit more than I can chew.
<bigjools> IBugSet.getByNumbers()
<bigjools> or just .get()
<nigelb> bigjools: where does it live?
<jtv> henninge: time to review?  https://code.launchpad.net/~jtv/launchpad/pre-824499/+merge/71536
<bigjools> lp/bugs/ ? :)
<bigjools> you need to install bzr grep
 * nigelb installs
<nigelb> hm, don't see anything like that.
<bigjools> bzr-grep I think is the package
<nigelb> ah, maverick+
<nigelb> I'm on lucid.
<bigjools> tsk :)
<nigelb> lp's deployed on lucid! So its a technically valid excuse :P
<bigjools> look in lib/lp/bugs/interfaces/bug.py
<bigjools> there is a pattern to our files y'know :)
<nigelb> heh, yeah. I should have guessed :)
<bigjools> there's a PPA with bzr-grep in it somewhere
<bigjools> it's a cracking plugin
<wgrant> bzr branch lp:bzr-grep ~/.bazaar/plugins/grep
<bigjools> I wonder if we can give nigelb the PyCharm licence key
<bigjools> maybe not, he doesn't have commit access
<nigelb> To LP?
<bigjools> yeah
<nigelb> I thought that's only for Canonical employees?
<bigjools> if you commit to other OSS projects you could get your own licence
<bigjools> indeed
<wgrant> I've given up on PyCharm for now.
<bigjools> why?
<StevenK> wgrant: Oh?
<nigelb> I like vim tbh.
<bigjools> I have the vim plugin for PyCharm
<bigjools> it's almost like using vim except you get something from the 21st century
<nigelb> heh
<bigjools> but seriously, it's kinda nice having a decent IDE
<nigelb> how does the Interface thing work?
<nigelb> standard OOP?
<bigjools> e_context
<nigelb> Like, where do I see the function defitions for fuctions in Ibug
<lifeless> s/interfaces/model/
<nigelb> Oh.
<bigjools> well he can't call stuff that's not in the interface
<bigjools> the interface is the best place to look
<bigjools> it also has docstrings
<bigjools> nigelb: look in the file I pasted earlier and search down for IBugSet
<bigjools> class IBugSet
<nigelb> Yeah, I was looking at that.
<henninge> jtv: looking
<henninge> jtv: why does "rsyncBackupDists" need a distribution parameter? I see no change in the method body.
<jtv> henninge: it becomes necessary for accessing configs later, in the follow-up branch.
<henninge> ok
<jtv> (self.configs becomes indexed by distro)
<henninge> jtv: ah yes, I see there are more lilke that!
<nigelb> bigjools: (maybe a stupid question), does IbugSet.get() correctly take care of hiding private bugs from the person requesting?
<nigelb> (the docstrings only tell baout bug not found. Not about bug found, but you can't see it situation)
<bigjools> nigelb: no idea, I've never done any bugs work
 * bigjools would hope that the security adapter does that
<nigelb> *whee* grep time :)
<bigjools> check in the security.py
<nigelb> Nothing about IBugSet in there
<stub> So we don't want to land code changes on db-devel, yet there are some db changes that require code updates. So the best way of handling this is wranging two branches? One launchpad/stable branch with the code changes, and merge this to a launchpad/db-devel branch with the schema change, and run tests for both branches and land separately?
<stub> (yes, pipes should make this workable)
<wgrant> stub: DB changes aren't allowed to require code updates.
<jtv> stub: I think we _have_ to make the code and db changes work independently.
<wgrant> stub: Or if they do, it must already be landed on devel and deployed.
<stub> I'm adding a NOT NULL constraint. That requires code to make sure it is never NULL.
<jtv> Not sure how renaming columns works.
<wgrant> It doesn't.
<stub> wgrant: Yes. I'm thinking about how to best wrangle this.
<wgrant> Well, you could do it with triggers.
<wgrant> Same as renaming tables.
<wgrant> stub: What needs wranglement?
<stub> I need to land code changes so I can land db changes and keep the test suite passing.
<stub> I'm adding a NOT NULL constraint. Some tests fail. I need to fix these tests on launchpad/devel.
<wgrant> The code needs to cope with both old and new schemas.
<wgrant> The code lands on devel. Once it's merged to db-devel, you can land the schema change.
<stub> Yes.
<wgrant> The code change is then deployed.
<wgrant> fastdowntime happens later.
<wgrant> Then it's all done.
<stub> I'm just wondering if the method of wrangling the branches is most efficient.
<stub> I need to do coordinated landing of changes to launchpad/devel and launchpad/db-devel.
<henninge> jtv: that's an almost mechanical change ... ;-) r=me
<wgrant> stub: Why can't you make the devel changes first?
<stub> I am!
<wgrant> I mean, at an arbitrary earlier point.
<stub> Yes, I sketched out a method of coordinating this.
<jtv> henninge: yes, I wanted to keep that separate from the less-mechanical changes.  Thanks!
<stub> My code changes are defined as 'what code changes are needed to allow this database patch to land'
<lifeless> stub: so you need to sequence it
<lifeless> stub: a) land code that makes sure its never NULL
<lifeless> let that get deployed.
<lifeless> then land the schema constraint
<stub> Yes. As I said before, I'm wondering on the best way of wrangling the branches.
<lifeless> then deploy that, and merge the db side of it to devel after the deploy
<stub> I'm thinking a pipe branch, first pipe from launchpad/stable containing code changes.
<stub> Second pipe from launchpad/db-devel containing the db-changes
<lifeless> yes, that should work fine
<stub> Run tests for both
<lifeless> I'd probably just do separate branhces
<lifeless> but thats me :P
<wgrant> lifeless: Speaking of you, when are we likely to lose you?
<wgrant> Due to the impending lifeless-ng.
<lifeless> wgrant: 38w1d
<wgrant> Eep.
<stub> The important things seem to be code branch from stable (as this will be merged into the db changes branch, so we don't want devel leaking in there), and to run the tests for code changes (don't break trunk) and code changes + db patch (the changes actually let me land the patch)
<lifeless> wgrant: normal births occur from 37w0d to 42w0d
<wgrant> I know that much :)
<lifeless> stub: landing separately will naturally run the changes separately :)
<stub> yes
<lifeless> wgrant: I don't know what they teach at aussie schools ;P
<wgrant> Hah.
<nigelb> lifeless-ng?
<stub> in this case, knowing what code changes are necessary requires running the test suite with the dbpatch
<nigelb> bugid is er what type of variable?
<lifeless> stub: I would be lazy; make the db branch, submit to ec2, if it lands \o/ if it doesn't, the error list will tell you what you need to change :)
<stub> lifeless: So skew in LaunchpadDatabaseRevision might actually be a good thing, as it gives us a little transition state during the period when patches are deployed on some databases and not others. Or it might just be a footgun.
<lifeless> stub: depends on what you mean by skew
<lifeless> stub: with transactions we have skew between slaves
<lifeless> stub: but never between the version on a slave, and the schema on the slave.
<lifeless> stub: what I meant by skew was a schema that doesn't match LDR [and not just an index existing]
<lifeless> ! http://mp3fs.sourceforge.net/
<stub> Need to think about it anyway.
<nigelb> I have an oops from my local instance, how I see the full OOPS?
<nigelb> (its an XHR call, so I can't actually see the page)
<stub> That FUSE idea is an interesting solution.
<lifeless> nigelb: /var/tmp/lperr/*/*
<nigelb> lifeless: thanks!
<nigelb> I may have run into a security problem. Gotta try and confirm.
<lifeless> nigelb: what are you hacking up ?
<nigelb> lifeless: making the linkage of bugs similar to branches, greyed out if the bug doesn't exist
<lifeless> nigelb: uhm
<lifeless> nigelb: do you mean linkification ?
<nigelb> yeah
<nigelb> bug 4595 actually I think
<lifeless> so, a few tips
<_mup_> Bug #4595: Don't auto-linkify non-existent bug reports <easy> <lp-web> <tales> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/4595 >
<lifeless> hah, whoever claimed easy is probably lying
<nigelb> I agree
<nigelb> I touched javascript and some python, not easy, but fun :)
<nigelb> I've got it down to harvesting, posting the bug numbers via xhr call, returning invalid bug numbers.
<lifeless> are you  doing it via a webservice call post lost ?
<nigelb> I'm doing it via an xhr call to +check-links
<nigelb> I almost got it  I think :)
<nigelb> (except for a possible security concern)
<nigelb> \o/ It works
<stub> Bah. Need separate branches or db-devel ends up in the code changes. Duh.
<nigelb> lifeless: It works \o/ Sadly, the I run into the security issue I feared.
<jtv> henninge: here's the follow-up to my earlier branch, if you still have time: https://code.launchpad.net/~jtv/launchpad/bug-824499/+merge/71544
<henninge> jtv: time? who owns time?
<jtv> henninge: Not saying you have to own it.  You might be holding someone else's, for instance.
<henninge> jtv: I won't be able to start for another hour or so, I am sorry.
 * henninge is off to lunch
<jtv> henninge-lunch: OK â I may be gone when you get back.
<nigelb> who has worked on bugs that I can take help of?
<lifeless> nigelb: most folk
<nigelb> lifeless: oh.
<nigelb> I just need some help trying to make sure I show only bugs that are visible to the current user, not just valid.
<nigelb> IBugSet.get() only checks if I have e permission.
<lifeless> so, I would use search
<lifeless> not BugSet.get
<lifeless> because you don't want to check every bug one sql line at a time
<lifeless> thats pretty much a guaranteed timeout right there
<lifeless> you can see how to do a narrow search by id if you poke around the bugbranch link code
<jelmer> wow, that +localpackagediff page is awesome
<bigjools> BugSet.getByNumbers() is what I recommended earlier
<bigjools> jelmer: :D
<nigelb> lifeless: ouch, branches is done that way.
<nigelb> or so I think.
<nigelb> Also, bug branch link code?
<nigelb> I don't know what that is to look for it
<lifeless> bigjools: that will perform horribly
<lifeless> nigelb: lib/lp/code/model/branchcollection.py line 408 has some related code
<lifeless> that might be enough to get you rolling
<nigelb> *looks*
<lifeless> bigjools: (the lookup would be fast, but every private bug would trigger separate just-in-time subscription lookups)
<bigjools> I was only asked about lookup
<lifeless> bigjools: I know
 * bigjools heads to lunch for real
<lifeless> adeuring: grahhh regex?!
<lifeless> adeuring: just use iso8601
<adeuring> lifeless: can you explain a bit?
<lifeless> bin/py -m pydoc 'iso8601.parse_date'
<lifeless> adeuring: rather than using a regex, use pre-existing code.
<lifeless> adeuring: if someone messes with a memo, they can get a UDF and we don't have to care
<adeuring> lifeless: a, thanks. I did not know about this module
<lifeless> its verra handy :)
<adeuring> yeah
<lifeless> I recently added it to our deps, as part of the oops extraction I deleted a very similar regex.
<jtv> I thought I could run the Initialize DistroSeries jobs with "./cronscripts/process-job-source.py -vv initializedistroseries" but that fails with "No section key named module."  Was a change in process-job-source configuration not applied to this config, or am I running the wrong thing?
* benji changed the topic of #launchpad-dev to: Das Thema fÃ¼r #launchpad-dev ist: https://dev.launchpad.net/ | On call reviewer: henninge, benji | Critical bugs: 238 - 0:[#######=]:256
<nigelb> lifeless: I'm confused. I don't understand how this works.
<lifeless> you crate a bugtasksearch params for the user, no status mask, and rather thank saying linked_branches=... you say bug=any(*ids)
<lifeless> then searchIds which will give you the actual visible ids
<lifeless> and you're done
<nigelb> where *ids is my list of ids?
<lifeless> yes
<nigelb> ok, more clear.
<nigelb> One more question :)
<nigelb> How do I get current user?
<lifeless> depends
<lifeless> :P
<nigelb> oh oh
<lifeless> if this is a web api, through partial function application
<lifeless> have a look at the interfaces/* files and look for user parameters with binding decorators
<nigelb> lazr.restful.declarations.REQUEST_USER?
<lifeless> something like
<lifeless> its 0020
<lifeless> so I am going to halt()
<nigelb> wait, what?
<nigelb> I didn't understand that :(
<benji> nigelb: here's an example: lib/lp/registry/interfaces/person.py, line 1724
<nigelb> benji: Ah. I kept seeing that in my grep, didn't realize that's exactly what I wwas looking for.
<nigelb> Thanks!
<benji> no problem
<adeuring> lifeless: fancy a review of a new version of my MP?
<adeuring> henninge: could you have a look at this mp: https://code.launchpad.net/~adeuring/launchpad/bug-739052-5/+merge/71548 ?
<henninge> adeuring: wow, that's long
<henninge> adeuring: and it has text conflicts
<adeuring> ouch.. should be 300 or 400 lines. let me check...
<henninge> adeuring: 1047
<henninge> benji: Hi! Are you working on a review from the queue?
<benji> henninge: not at the moment, I haven't quite spun up my reviews yet
<henninge> benji: ok, cool
<deryck> Morning, all.
<adeuring> morning deryck
<adeuring> henninge: could have a look again: https://code.launchpad.net/~adeuring/launchpad/bug-739052-5/+merge/71548 (201 lines)
<deryck> abentley, can you hear us?
<abentley> deryck: no
<deryck> abentley, ok, we don't hear you either.
<deryck> abentley, we could do skype if skype works for you.
<abentley> deryck: let's try that.
<deryck> abentley, ok, adeuring is switching machines and I'll call everyone.
<abentley> deryck: restarting.
<abentley> deryck: actually, I'll use skype on my phone...
<deryck> abentley, ok, that works.
<deryck> abentley, I have your number in my contacts it seems.
<deryck> adeuring, you ready now?
<adeuring> deryck: skype is starting...
<abentley> deryck: I have skype on my phone.
<adeuring> deryck: I am online on  kype
<deryck> abentley, ah, ok.  gotchas.
<abentley> deryck: try again please.
<deryck> abentley, ringing....
<abentley> deryck: I keep hitting the answer button, but it doesn't answer even though it shows it knows I've hit the button.
<deryck> hmmm
<deryck> and now we lost adeuring.
<adeuring> yeah... connection dropped... no idea why
<deryck> abentley, shall I call your land line?
<deryck> abentley, or whatever number I have listed as "real phone" :)
<abentley> deryck: no, phone is haywire.
<deryck> abentley, ah, ok.
<abentley> after exiting Skype, it's still ringing.
<abentley> deryck: rebooting.
<abentley> deryck: back
<deryck> abentley, ok, dialing you in....
<nigelb> Hi, could someone help with BugTaskSearchParams?
<nigelb> Can I give it a list of bugs?
<nigelb> Or rather, how do I give it a list of bugs.
<benji> nigelb: I've never used BugTaskSearchParams, but lib/lp/bugs/browser/bug.py, line 558 looks exemplary
<nigelb> benji: Interesting. That's what I first tried and ran into trouble.
<nigelb> Let me try it again so that I can capture the error message
<nigelb> benji: yup, getting hit with "TypeError: any() takes exactly one argument (2 given)"
<benji> nigelb: it sounds like you're using the builtin "any", not the one imported from canonical.launchpad.searchbuilder
* henninge changed the topic of #launchpad-dev to: Das Thema fÃ¼r #launchpad-dev ist: https://dev.launchpad.net/ | On call reviewer: benji | Critical bugs: 238 - 0:[#######=]:256
<nigelb> benji: Ahhh. Yes. Thanks :)
<nigelb> (I didn't know there was one to import :p)
<henninge> adeuring: r=me
<adeuring> henninge: thanks!
<nigelb> Is there a way I can use pdb for debugging launchpad stuff?
<benji> nigelb: re. pdb: yes; the easiest thing is to put "import pdb;pdb.set_trace() what is the context?
<benji> oops
<benji> nigelb: re. pdb: yes; the easiest thing is to put "import pdb;pdb.set_trace()" in your code.  What is the context?
<nigelb> benji: I tried that. Didn't work.  its a webservice call.
<benji> nigelb: define "Didn't work."
<nigelb> well, I didn't get dropped to a ppb shell
<nigelb> *pdb
<nigelb> and the xhr request was stuck waiting for a response
<benji> nigelb: did the server process have its stdio connected to a terminal?
<nigelb> benji: I just use make run
<nigelb> which I'm not sure leaves stdio
<benji> nigelb: I'm not certain it matters, but try bin/run instead
<nigelb> benji: No, not in a pdb shell this time either.
<nigelb> Is there a better way to debug stuff that call XHR?
<benji> that's odd; I don't think I've ever seen that symptom
<benji> re. better way: generally having a test that excersizes the misbehaving code would be easier (both for pdb-based debugging and otherwise)
<nigelb> hmm, yes. that should definitely leave me in a shell, yes!
<statik> nigelb, fwiw I usually use winpdb when debugging any python web services so I don't have to worry about having the server connected to a terminal
<nigelb> statik: oh? let me read up on that
<statik> nigelb, the first code snippet on this page http://winpdb.org/docs/embedded-debugging/ is what you would need.
<nigelb> statik: thank you! I've been at this almost all-day :)
<statik> then you connect from the gui debugger
<statik> np, hope it helps.
<abentley> deryck: pre-imp call?
<deryck> abentley, sure.  Just got off a call.  Can I have 10 minutes to get coffee, rest my voice? :)
<abentley> deryck: sure.
<deryck> abentley, I'll ping shortly.
<deryck> abentley, I'm ready now.  mumble or skype?
<abentley> mumble
<LPCIBot> Project devel build #975: FAILURE in 27 min: https://lpci.wedontsleep.org/job/devel/975/
<abentley> deryck: https://bugs.launchpad.net/launchpad/+bug/820039
<_mup_> Bug #820039: process-mail.py fails with a LookupError: unknown encoding: macintosh error <mail> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/820039 >
<nigelb> statik: winpdb is simply awesome :)
<bigjools> nigelb: or pycharm "just works" for all this debugging :)
<nigelb> bigjools: hah, stop tempting me ;)
<bigjools> nigelb: it has a free 1 month trial
<bigjools> oh sorry that's not stopping tempting you
<nigelb> hehe
<bigjools> nigelb: my productivity went up tenfold when I started using pycharm
<bigjools> oh sorry that's not stopping tempting you
<nigelb> Is there a possibility that I can get commit access to LP without being employed by Canonical?
<bigjools> nigelb: none at all, sorry
<bigjools> for various $LEGAL_REASONS
<nigelb> Darn. I guess I should just wait to see if you guys will hire me :)
<bigjools> heh
<bigjools> we hired the last prolific contributor
<jtv> henninge: before I head outâ¦ any questions about the review?
<nigelb> yeah, but beating *that* contributor is probably going to take about 10 years
<henninge> jtv: oh, I assumed you had already gone.
<jtv> Tsk.  :)
<henninge> jtv: I am writing the review right now. All fine.
<jtv> Great, thanks
<henninge> just a few suggestions
<jtv> Even better.
<henninge> jtv: sent
<jtv> thanks!
<henninge> jtv: do you know how MatchesStructure works?
<jtv> Yes, but attempts to find it back failed for some reason.
<henninge> "find it back" ?
<henninge> oh
<henninge> jtv: it's in testtools.matchers
<jtv> I thought I'd looked there.
<henninge> jtv: it is missing from __all__ ...
<jtv> That'd do it.
<jtv> I don't see it in help(testtools.matchers).
<jtv> Meanwhile, the MP hasn't updated yet.  Strange.
<jml> jtv, henninge: fixed in forthcoming release
<jtv> Great, thanks.
<henninge> jml: yes, just checked that ;-)
<jml> Also, http://testtools.readthedocs.org/en/latest/for-test-authors.html is quite complete
<henninge> jml: I am eager for that release, MatchesStructure.byEquality is really missing ... ;)
<jml> henninge: nothing to stop LP from using a snapshot of trunk
<henninge>  I guess ...
 * henninge takes a note to look into that tomorrow
<jml> basically, as soon as https://bugs.launchpad.net/testtools/+bug/804127 is fixed and the other critical bug fix lands, I'm going to do a release
<_mup_> Bug #804127: assertThat(..., verbose=True) sometimes generates unprintable AssertionErrors <unicode> <testtools:In Progress by jml> < https://launchpad.net/bugs/804127 >
<jtv> bac: if you're still up for it, I resubmitted my MP at https://code.launchpad.net/~jtv/launchpad/bug-659769/+merge/71505
<bac> jtv: i didn't get that far into it.  might be best to ask the ocr
<jtv> OK
<bac> jtv: did you figure out how it got messed up?
<nigelb> oh dear. I'm through through a whole bunch of Storm.
<jtv> There was a rollback in devel somewhere inbetween.
<bac> jtv: ah
<jtv> henninge: not a trace of your review, in email or MP.  Are you sure it went in?
<henninge> hm
<henninge> argh
<henninge> "bad signature"
<henninge> I'll paste
<jtv> Grr
<henninge> jtv: pasted
<jtv> loadingâ¦
<jtv> Got it!
<jtv> Thanks.
<henninge> jtv: good night!
<jtv> nn
<nigelb> bigjools: heh, I respectfully surrender before Launchpad. I think after 9 hours I'm going crazy debugging ;)
<jtv> MatchesStructure.byEquality, MatchesStructure.from_example.  The great thing about conventions is that there are so many to choose from!
<jml> jtv: please file a bug. we're pretty darn responsive
<jtv> Looks like it's already been fixed.
<jtv> Ironically, it was the documentation that led me down the wrong path there.
<bigjools> nigelb: heh, good luck tomorrow
<nigelb> bigjools: I just realized wallyworld_ offered to mentor me for this. I can now bug him :P
<jtv> jml: MatchesStructure.fromExample does work.
<jml> Fixed.
<jtv> Cool.
<bigjools> nigelb: do it
<nigelb> bigjools: I'm writing out an email ;)
<nigelb> loggerhead having slight issues? Throwing up errors once or twice
<sinzui> jcsackett, do you have time to mumble? I need help http://pastebin.ubuntu.com/666686/
<jcsackett> sinzui: i can mumble now.
<abentley> benji: could you please review https://code.launchpad.net/~abentley/launchpad/macintosh-encoding/+merge/71599 ?
<benji> abentley: sure, I can right now
<benji> abentley: looks good, I thought of one teeny thing you might want to do in addition
<abentley> benji: sure.
<LPCIBot> Project db-devel build #811: FAILURE in 6 hr 9 min: https://lpci.wedontsleep.org/job/db-devel/811/
<lifeless> morning
<nigelb> lifeless! Morning!
<sinzui> lifeless, do you have time what happens when a bug is made private http://pastebin.ubuntu.com/666686/
<lifeless> time <missing verb> what. .. ?
<elmo> lifeless: s/do/if/ s/time/time:/
<sinzui> lifeless, do you have time to discuss what happens when a bug is made private http://pastebin.ubuntu.com/666686/
<lifeless> sinzui: certainly! voice or irc ?
 * elmo guess wrong ;)
<sinzui> skype?
<sinzui> oh, wrong bug in paste too http://pastebin.ubuntu.com/666767/
<lifeless> signing in now
<lifeless> elmo: nice try though :)
* benji changed the topic of #launchpad-dev to: Das Thema fÃ¼r #launchpad-dev ist: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 238 - 0:[#######=]:256
<lifeless> sinzui: bug 405277
<_mup_> Bug #405277: Private teams are not able to join other teams (public or private) <disclosure> <lp-registry> <Launchpad itself:Triaged> < https://launchpad.net/bugs/405277 >
<mtaylor> httplib2.SSLHandshakeError: [Errno 1] _ssl.c:499: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
<mtaylor> EEK?
<mtaylor> http://paste.openstack.org/show/2175/ <--- this works on my laptop - but not on another server
<mtaylor> lifeless: ^^ ?
<mtaylor> lifeless: (also, good morning!)
<lifeless> mtaylor: hi
<lifeless> mtaylor: oneiric ?
<mtaylor> lifeless: natty on the server (where it doesn't work) maverick on my laptop
<lifeless> got ca certs installed ?
<mtaylor> uhm.
<mtaylor> lifeless: shouldn't that just be something that's there?
<lifeless> mtaylor: p'rhaps
<jeblair> ii  ca-certificates                      20090814+nmu2                            Common CA certificates
<mtaylor> lifeless: what jeblair said
<lifeless> ok, thats current in natty
<lifeless> do you have the curl ssl update ?
<lifeless> (libcurl3, python-pycurl)
<lifeless> and are you behind a ssl mitming firewall ?
<mtaylor> lifeless: it shouldn't be
<mtaylor> python-pycurl needs installed
<lifeless> (don't)
<lifeless> that excludes another bug
<lifeless> uhm
<lifeless> 'its not LP' :P
<mtaylor> oh. crud. should uninstall?
<mtaylor> :)
<lifeless> maverick -> natty was a bump launchpadlib API change
<lifeless> its possible thats the cause
<lifeless> bumpy
* lifeless changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 238 - 0:[#######=]:256
<mtaylor> lifeless: well, I mean - I've used launchpadlib on this box before
<lifeless> mtaylor: sure
<lifeless> mtaylor: I've no particular insight; there are only two servers you can be talking to (the dual apache SSL unwrappers)
<lifeless> mtaylor: if either of those was causing browsers to fail validation we'd be innundated
<lifeless> mtaylor: so I strongly suspect something local to that box/network.
<mtaylor> lifeless: ok
<lifeless> jelmer: bug 826082 - do you want to roll your function combination back, or fix it up ?
<_mup_> Bug #826082: BadUrl from safe_open opening a branch <oops> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/826082 >
<StevenK> lifeless: O hai. Can I get your opinion on https://code.launchpad.net/~stevenk/launchpad/testfix-ftpmaster/+merge/71619 ? I'm not really across testtools matching things.
<lifeless> what about it ?
<lifeless> the testtools spelling changed between initial landing and release
<StevenK> Ah, right
<jelmer> lifeless: Thanks for the analysis. I'll fix it up.
<lifeless> jelmer: cool, thanks.
<lifeless> jelmer: and it was all jam :)
#launchpad-dev 2011-08-16
<mwhudson> jelmer: the diff on  https://code.launchpad.net/~jelmer/launchpad/bzr-code-imports/+merge/70684 *still* seems to have extraneous changes
<mwhudson> jelmer: which must be quite frustrating by now :)
<mwhudson> jelmer: ah, seems you merged db-devel?
<jelmer> mwhudson, yeah, as it has the bzr-2.4b4 branch merged, which is targetted to db-devel
<jelmer> mwhudson, let me resubmit
<lifeless> pop quiz
<lifeless> who cares about informational oopses?
<mwhudson> lifeless: people presumably care about the ones made with ++oops++
<lifeless> mwhudson: I don't think they would suffer if that was not 'informational'
<lifeless> jelmer: does the code change depend on schema changes that are not yet deployed ?
<jelmer> lifeless: No, but it has merged some revisions from an older revision of db-devel which seems to confuse the MP code
<lifeless> so we don't deploy from db-devel
<lifeless> -ever-
<lifeless> -at all-
<lifeless> landing it on db-devel will just cause conflicts and pain
<jelmer> ok
<lifeless> (conflicts when folk changing devel conflict with your code)
<lifeless> (pain when conflicts happen :P)
<lifeless> is it perhaps triggering a criss-cross merge case?
<lifeless> what happens when you merge (but don't commit) devel into it ?
<jelmer> conflicts :P
<lifeless> jelmer: so, I would fix that :P
<lifeless> and propose to devel
<lifeless> mwhudson: I'm thinking to nuke the 'informational' flag
<lifeless> mwhudson: and just use specific exceptions like 'ProfilingOops' instead to filter them on the reporting side
<mwhudson> lifeless: ah i see
<lifeless> mwhudson: things like the hwdb code that logs an oops when it gets a POST missing a field
<lifeless> suggest that the informational side has been misunderstood and misused
<wgrant> lifeless: I think it is useful to have an OOPS-like mechanism for non-critical logging.
<lifeless> wgrant: well, I'm going through our uses
<lifeless> wgrant: so far I've found one case that makes sense - profiling / ++oops++ - both of which use a custom exception
<lifeless> the disconnected handling in publisher, for instance - that deserves to be a real oops
<lifeless> its a problem if its happening
<lifeless> (it should never be happening)
<lifeless> yep, all uses were either a) noddy or b) trivially classified on the exception anyway.
<lifeless> *nuke*
<sinzui> wallyworld_, did you need me to look at something?
<wallyworld_> sinzui: yes, just to approve the ff request: https://wiki.canonical.com/InformationInfrastructure/OSA/LaunchpadProductionStatus#Feature_Flag_Changes
<wallyworld_> thanks
<LPCIBot> Project devel build #976: STILL FAILING in 6 hr 7 min: https://lpci.wedontsleep.org/job/devel/976/
<lifeless> jelmer: lol; meep - your branch is really confused isn't it :)
<wallyworld_> StevenK: want a trivial review? https://code.launchpad.net/~wallyworld/launchpad/remove-answer-contact-827051/+merge/71632
<nigelb> wallyworld_: heya
<wallyworld_> nigelb: hi, got your email, still catching up with things, sorry haven't replied yet
<lifeless> speaking of reviews
<lifeless> wallyworld_: still pending on loggerhead;sorry
<lifeless> https://code.launchpad.net/~lifeless/python-oops-datedir-repo/extraction/+merge/71633 and https://code.launchpad.net/~lifeless/launchpad/useoops/+merge/71634
<wallyworld_> lifeless: np. i was going to remind you later today :-P
<lifeless> are the next stage of my ripping oops out of LP
<nigelb> wallyworld_: great thanks :)
<wallyworld_> nigelb: will look very shortly
<nigelb> \o/
<lifeless> I think the only thing left is an http module and either a canonical module or some sensible-defaults copied into the core oops system.
<jtv> Any reviewers in the house?  Got a job for you!  https://code.launchpad.net/~jtv/launchpad/bug-826659/+merge/71636
<jtv> Short diff, long cover letter because I vent some steam and formulate some rules for testing.
<wallyworld_> jtv: i'll look if someone can mentor
<jtv> hi wallyworld_!  Thanks.  I see a slight conflict of interest with me doing that, obviously.  :)
<wallyworld_> jtv: yep :-) i and looking at something for nigelb just now but will do the review asap after that
<jtv> Thanks.  Let me know if you need any mentoring for branches other than mine.  And maybe it's time to look at graduation â I have a feeling we may have lost our procedure for that.
<wallyworld_> nigelb: your @call_with(user=REQUEST_USER) is having no effect since it's in the wrong place.
<wallyworld_> nigelb: you need to add this to an interface method and register the interface with the web service parsing infrastructure so it gets slurped up
<nigelb> wallyworld_: oh. *thats* why.
<wallyworld_> nigelb: there's various webservice.py files in the code base which is where the webservice stuff looks to see what it needs to register
<wallyworld_> with your code at the moment, user is set to None
<nigelb> ah, that explains that.
<wallyworld_> you could pick an example interface what is exported to the web service and model what you need to do on that
<nigelb> so, write a wrapper over the search function using a webservice interface?
<wallyworld_> you also should write a test for the web service side of it
<nigelb> I should write a javascript test too. I've been lazy ;)
<wallyworld_> nigelb: you don't need to write a wrapper - you just need to figure out how to get the current logged in user inside your check_bug_links method.
<nigelb> ah!
<wallyworld_> your code looks ok so long as you know the user
<wallyworld_> we uased to use ILaunchbag to arbitarily get the user for a given interaction but that's frowned upon now
<StevenK> That's because ILaunchBag needs to die horribly.
<StevenK> And painfully.
<wallyworld_> the @call_with is one way of doing it if the method is an existing web service
<wallyworld_> there may be a better way without having to export teh method as a web service but i'm not sure off hand what that would be
<nigelb> this has been done before? and I just need to find some example code and work my code the same way?
<wallyworld_> StevenK: yes, agree about ILaunchbag. do you know any other way to get the logged in user other than @call_with ?
<wallyworld_> nigelb: @call_with is used in many places
<nigelb> wallyworld_: heh, except I didn't understand the correct way of using it :)
<wallyworld_> if you are in a view, you can type self.user i think. but we are not in a view here
<wallyworld_> i don't know enough about the zope machinery to understand how the user is extracted from the request
<nigelb> the only place call_with works is in interfaces?
<wallyworld_> nigelb: LinkChecker is initialised with the request object
<wallyworld_> nigelb: i think you can get the user from in there
<wallyworld_> nigelb: afaik, all web service decorators need to be added to interface methods
<nigelb> ah, cool.
<wallyworld_> nigelb: try user = request.principal.person
<wallyworld_> not sure if that's kosher but it works
<nigelb> heh, okay :)
<wallyworld_> nigelb: for anonymous requests, i suspect principal will be None so check for that
<nigelb> sure, will do
<wallyworld_> so forget the web service stuff
<mwhudson> i think IPerson(request.principal) might be more kosher
<lifeless> wallyworld_: @call_with for web service methods/properties is best practice.
<wallyworld_> mwhudson: ah, thanks
<wallyworld_> lifeless: it's not exposed as a web service method atm
<wallyworld_> otherwise @call_with would be the best solution i agree
<wallyworld_> nigelb: see mwhudson's comment above - use IPerson(request.principal)
<nigelb> wallyworld_: already changed code :)
<wallyworld_> excellent
<lifeless> wallyworld_: ah, I didn't think it could be anything -other- than a web service clal
<lifeless> wallyworld_: :)
<wallyworld_> lifeless: it's defined in zcml as a browser:page implementation
<wallyworld_> and the __call__ method is implemented
<lifeless> yeah, I can imagine
<wallyworld_> web service is sort of overkill for what it does perhaps, it's only an "internal" api
<lifeless> in principle the web service would take care of encoding, auth checking, marshalling etc
<lifeless> doing it by hand is, well, by hand.
<nigelb> err, what's the result of a searchBugIds call? I don't seem to be able to understand this.
<nigelb> Here's the call I'm making and its associated traceback http://paste.ubuntu.com/666969/
<lifeless> a list of ids :)
<lifeless> nigelb: your bug_ids set is full of strings.
<lifeless> nigelb: also, that python loop will be -very- slow
<lifeless> nigelb: you need to map it into a set -
<lifeless> found_ids = set(bug_ids)
<lifeless> missing_bugs = found_ids - bugs
<nigelb> oh.
<nigelb> I didn't know I could do that :|
<lifeless> nigelb: I wouldn't use an english description either, you want minimal data flowing - this is an implementation detail, not a UI
<lifeless> so the thing you serialise out should just be the ids of the inaccessible bugs, no explanation why
<nigelb> lifeless: the why explanation shows up as tooltips.
<lifeless> nigelb: there is only one explanation
<nigelb> that's how the javascript end of things seem to be configured.
<lifeless> nigelb: sure, but you can do that on the client.
<wallyworld_> jtv: you and your fancy latin words
<lifeless> nigelb: disclosing whether the bug was private or just a gap in the bug sequence would leak information about the bug.
<nigelb> heh, that means creating a new javascript function on the client ;)
<jtv> wallyworld_: I'm sorry if I urinated you off, amice
<lifeless> nigelb: well, up to you where you do it; just saying what I see ;)
<wallyworld_> jtv: no, you didn't, just saying :-P
<jtv> :)
<nigelb> lifeless: heh, my first non-trivial branch, so I think I might aim for perfection or as much close to it as I can get.
<lifeless> nigelb: as long as its fit for purpose
<lifeless> nigelb: beyond that, shrug :)
<nigelb> heh
<nigelb> lifeless: for some reason the missing_bugs = found_ids - bugs isn't working
<lifeless> nigelb: break in with pdb and have a look around
<nigelb> in the middle of doing that
<nigelb> wait, what.
<nigelb> search returns bug ids as int.
 * nigelb cries
<nigelb> is there a way to convert that to string or make it return strings?
<lifeless> nigelb: like I said, your thing was full of strings
<lifeless> nigelb: no, you should be converting your strings to ints before you query in fact ;)
<nigelb> lifeless: bug numbers can't exceed the length of ints, can't they?
<lifeless> [because the data type is int]
<lifeless> nigelb: do you mean as in MAXINT ?
<nigelb> yeah
<lifeless> we can upgrade the column to 64-bit before that
<lifeless> but we'll be a few decades yet
<nigelb> oh. ok.
<nigelb> IT WORKS \O/
<StevenK> lifeless: So I should write a loop to create 200,000 bugs to help it along?
<lifeless> StevenK: ...
<lifeless> poolie: around ?
<StevenK> lifeless: :-P
<poolie> hi lifeless, otp atm
<poolie> to statik
<lifeless> kk
<lifeless> wgrant: you are terrible at leave.
<wallyworld_> lifeless: like you can talk
<StevenK> Haaha
<StevenK> wallyworld_ is right, you so can't.
<wallyworld_> mr pot, meet mr kettle
<lifeless> wallyworld_: I'm talking from a position of knowledge.
<lifeless> so I'm thinking about oopses here
<wgrant> lifeless: And I've just fallen into the trap of correcting incorrect question answers.
<lifeless> wgrant: i hope you take time in lieu :)
<wgrant> Hah.
<wgrant> It will only take a few minutes.
<lifeless> cognitive overhead :)
<lifeless> I *know* it takes me a fortnight +- to spin down from work, so I plan leave accordingly
<wallyworld_> that's a loooong time
<lifeless> wallyworld_: yes, and if you think back to last december - after about two weeks I was around much less
<wgrant> Indeed.
<lifeless> wallyworld_: if I take < 2 weeks leave with the intent of relaxing, its an utter fail
<wallyworld_> lifeless: i was in NZ for most of Dec, so didn't notice :-)
<lifeless> wallyworld_: nice, did you drive around ?
<lifeless> wgrant: btw, talking about generic names
<lifeless> http://pypi.python.org/pypi/Products.Collage/1.3.6
<lifeless> wgrant: 'Products' as a top level package in python.
<wallyworld_> yes, started in Dunedin, all through south island, then onto north island
<wgrant> lifeless: Sounds like Zope.
<wgrant> Is it?
<lifeless> wgrant: plone, so more or less yes
<wgrant> I think Zope 2 is allowed to have the global namespace.
<wgrant> RIght.
<wgrant> Since Zope invented just about everything, and they use sensible namespaces now :)
<lifeless> so anyhow
<lifeless> I'm thinking about how to pass in things like the request object, urls, exc_info
<lifeless> the generic way to wire up for DI would be to supply a container callback
<lifeless> but this is going to be fuugly for the simple case
<lifeless> so I'm thinking of having a 'context' parameter to config.create()', and documenting that on_create hooks will receive this, and it can be used to psas things to the hooks, and some well known hooks will look for things there.
<lifeless> like - attaching exception info would look for context['exc_infp']
<poolie> lifeless: is the preforking server unblocked now? or almost?
<lifeless> poolie: not until we have the disconnect stuff done
<lifeless> wgrant: what do you think? The alternative seems to be tighter coupling between report population/creation and the point the report is created.
<lifeless> StevenK: wallyworld_: ^
<wallyworld_> lifeless: i'm not sure i have enough context - you want to do IOC with the new oops module?
<wallyworld_> to allow the oops module to poke stuff back into the container?
<lifeless> wallyworld_: I want it to play with IOC environments (like lp's) and with non IOC stuff (like LaunchpadScript, loggerhead, gpgverify)
<lifeless> wallyworld_: so here is an example case
<lifeless> in LP we use zope.exceptions.exception_formatter to format the traceback in an oops report
<lifeless> this is pretty zope specific
<lifeless> even though, thankfully, its a the edge of the dependency graph I think.
<wallyworld_> yes, i can imagine it would be zope specific
<lifeless> so oops *could* use that
<lifeless> and have config.attachException(report, exc_info)
<lifeless> this would drag in zope.exceptions and zope.interfaces
<lifeless> which isn't too bad
<wgrant> Did you know that imposing Zope dependencies aparent from zope.interface and zope.component is considered a crime against humanity?
<wgrant> s/aparent/apart/
<lifeless> wgrant: well this is why the thought going into this
<lifeless> an alternative would be to say 'if you want to attach an exception, do it yourself', but thats cruel.
<lifeless> a third way would be to have an explicit config knob 'call this function to configure the traceback formatter'
<wallyworld_> lifeless: i'd rather pass in some sort of context with redirects to the preferred implementation as you say
<lifeless> the way I am considering is to say 'put your exception in content['exc_info'], and your exception on-create hook will format
<lifeless> if you want a different formatter, don't put the supplyed on-create hook in there, instead put your own one.
<lifeless> supplied.
<lifeless> this kindof shifts the complexity around
<lifeless> but
<lifeless> I think it starts to pay off when you consider handling of 'request' objects
<lifeless> and the like.
<lifeless> where the core really shouldn't have any concept of such a beast.
<wallyworld_> i don't think it introduces too much extra complexity any simple case
<lifeless> so create(content=dict(request=current_browser_request(), exc_info=sys.exc_info(), program=sys.argv[0])
<lifeless> )
<lifeless> and let the hooks suck out what they want
<lifeless> I'm down to *one function* to pick apart
<lifeless> and then launchpadloggerhead can lose a canonical import, and gpgverify can work.
<wallyworld_> maybe a namedtuple rather than a dict?
<lifeless> wallyworld_: blah :)
<lifeless> wallyworld_: theres no particular reason to make it immutable
<wallyworld_> not so much to make it immutable but for type safety and extensibility
<wgrant> How's that more typesafe or extensible?
<lifeless> you'll need to expand on how it gives those in a way a dict would not ?
<wallyworld_> i guess you could subclass dict if you needed to
<StevenK> To do what?
<wallyworld_> add extra functionality besides name-values pairs to the context
<lifeless> wallyworld_: uhhh, you've *totally* lost me.
<lifeless> wallyworld_: so the point of the context is to pass data from outside 'create' to each on_create hook, without needing create-and-every-callbacj to take **kwargs
<wallyworld_> context could be more than just a container for name-value pairs- it couild actually provide some other encapsulated functionality
<lifeless> wallyworld_: it could; what do you have in mind ?
<wallyworld_> just a sec, otp
<wallyworld_> lifeless: i dont' have any specific functionality in mind for this case, but in practice such context objects can be used for more than data passing - they can provide useful behaviour such as access to container specific services
<wallyworld_> so a dict may be too simple
<lifeless> wallyworld_: so, I would say that if someone in a container environment needs access to the container, they do:
<lifeless> context=dict(container=realcontainer, exc_info=sys.exc_info(), ... etc)
<lifeless> wallyworld_: and have their custom on_create hook do content['container'].containerservicecall()
<lifeless> wallyworld_: or even implement __getitem__ on their container and pass it in directly, if the protocols line up well enough
<lifeless> wallyworld_: remember we're duck typed, so there is no need for it to *be* a dict, not even a subclass :)
<wallyworld_> that would work. i guess it comes down a little to personal preference. i tend to dislike using simple data structures as integration "glue" since often i find more is needed
<lifeless> I've come to really appreciate minimalism
<wallyworld_> but maybe duck typing removes the need for that as you say
<lifeless> anyhow, that can be iterated on
<lifeless> given we have 1(1!) user so far :P
<wallyworld_> sure, so +1 for option 3 :-)
<wallyworld_> however it's done, we just need to be clear on the contract between the modules and ensure that is satified from both sides
<wgrant> Now now, clear and sensible contracts are not the Launchpad way.
<poolie> lifeless: i'm off the phone but i might have some lunch now
<poolie> will ping you after
<wallyworld_> lifeless: quick question - what's the storm syntax for a query term checking for "is null" that I can pass to "where=And(*terms)"? instead of [MyClass.property == 'name'] I want [MyClass.property is null] or whatever
<lifeless> poolie: ok
<lifeless> wallyworld_: ==None
<StevenK> wallyworld_: == None
<wallyworld_> lifeless: StevenK: thanks!
<StevenK> wallyworld_: I think you've broken devel on buildbot
<StevenK> Failure in test lp.answers.tests.test_questiontarget.QuestionTargetAnswerContactTestCase.test_canUserAlterAnswerContact_owner
<wallyworld_> StevenK: i'll look
<wallyworld_> StevenK: ffs. i didn't remove a test when i did the revert earlier
<StevenK> FAIL
<wallyworld_> yes :-(
<StevenK> There's a JS test and a python test both failing
<wallyworld_> i'll check the js one
<StevenK> RARGH
<StevenK> Stupid librarian logs attached to test failures
<StevenK> USELESS
<wallyworld_> StevenK: which js test? bb only shows 1 failure, the py one
<wgrant> StevenK: You should fix the fixture to only attach the log for the current test.
<StevenK> wgrant: I should fix the fixture to not attach the log in the first place!
<StevenK> wallyworld_: Read the summary, there are two.
<lifeless> StevenK: attaching the log is great
<lifeless> StevenK: what needs to change is to only attach the *new entries* for a test.
<StevenK> Sure, great. 3,800 lines between the test failure and the traceback
<jtv> Any review mentors available?  https://code.launchpad.net/~jtv/launchpad/bug-826659/+merge/71636
<StevenK> Well. Bugger.
<StevenK> Calling the column sourcepackagename hurts when you join against SPR
<StevenK> :-(
<wallyworld_> StevenK: can you possibly +1 this? do I land with 'testfix' now or wait till bb barfs? https://code.launchpad.net/~wallyworld/launchpad/remove-tests-for-badrev-13696/+merge/71642
<jtv> wgrant: are you reviewing today?
<StevenK> jtv: wgrant is on holidays
<jtv> Then he shouldn't be here!
<wallyworld_> jtv:  *supposed* to be
<jtv> Ah yes
<StevenK> wallyworld_: r=me
<jtv> Well, I'll just go have lunch then or something.
<StevenK> wallyworld_: If you land it now, you may have to force buildbot when it finishes.
<jtv> wallyworld_: I added a follow-up note on my MP, even though I'm not formally mentoring that one.  :)
<wallyworld_> jtv: thanks, will look
<wallyworld_> StevenK: i'll be at soccer when it barfs but will force as soon as i return, about an hour later. so i'll land now so it's all ready
<lifeless> StevenK: since you're familiar with the code... https://code.launchpad.net/~lifeless/python-oops/extraction/+merge/71510 please?
<StevenK> lifeless: 1169 lines!?
<lifeless> StevenK: bah, thats the old one.
<lifeless> one sec
<lifeless> StevenK: https://code.launchpad.net/~lifeless/python-oops-datedir-repo/extraction/+merge/71633 is the one I meant.
<StevenK> lifeless: So, calling the new column on SPPH sourcepackagename leads to problems
<lifeless> StevenK: existing queries barf ?
<StevenK> Right
<lifeless> StevenK: lets use a different name; easier and safer than fixing all queries in advance.
<lifeless> spn_cache or something
<StevenK> lifeless: I've switched to 'name'
<lifeless> StevenK: I'd prefer something other than that, because of 'name' being the url component in so many other classes.
<lifeless> StevenK: however, up to you.
<StevenK> lifeless: 'spn' / 'bpn' ?
<lifeless> StevenK: sure
<lifeless> Subject: RETURNED MAIL: DATA FORMAT ERROR
<lifeless> heh
<StevenK> lifeless: r=me, with a slight wrist-slap
<lifeless> sexy times
<lifeless> ah yes, the version number.
<lifeless> that was oversight in the move before
<wallyworld_> StevenK: if you are around in about 1hr30 and force a build, i'll buy you a beer :-)
<StevenK> wallyworld_: You should have landed it with [testfix]
<wallyworld_> StevenK: bollocks. can i change it now?
<StevenK> No, it's landed.
<wallyworld_> bollocks x 2
 * wallyworld_ gotta run to soccer
<poolie> lifeless, hello?
<lifeless> can has reviews? tiny: https://code.launchpad.net/~lifeless/python-oops/extraction/+merge/71644 and getting big but its really simple stuff... just rearranged code: https://code.launchpad.net/~lifeless/launchpad/useoops/+merge/71634
<lifeless> poolie: hi
<lifeless> poolie: sure, if you want to ring the landline ?
<lifeless> poolie: ?
<LPCIBot> Project db-devel build #812: STILL FAILING in 5 hr 50 min: https://lpci.wedontsleep.org/job/db-devel/812/
<poolie> oh sorry, sure
<adeuring> good morning
<stub> http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/view/head:/lib/canonical/launchpad/webapp/dbpolicy.py?file_id=x_Andrew_Bennetts_%3Candrew.bennetts%40canonical.com%3E_Fri_Oct__1_11%3A57%3A48_2004_12702.0 is failing for me, but otherwise code browse was reasonably snappy
<stub> "redirecting the request for this address in a way that will never complete"
<lifeless> stub: bug 819444
<StevenK> stub: O hai. I've had to rename {binary,source}packagename to {b,s}pn, but you think an index on just that column is fine to start with?
<stub> Why the rename, apart from long names suck?
<stub> An index on just that column is fine. If you know the queries it is needed for, we might be able to make a compound index that satisfies those queries as well as the population job.
<StevenK> stub: Breaks existing queries that join against SPR and SPN
<stub> I'm not sure I like that rationale. We know we need to fix code when changing the schema.
<lifeless> stub: we can't be sure we've caught all the existing queries
<stub> I would go for 'because the long names suck, so we should just use spn and bpn everywhere' but that doesn't solve your problem.
<lifeless> stub: the reason for the breakage was a previously unique column name becoming non-unique + hand crafted queries not putting the table name in place
<stub> lifeless: You could say that about every new column addition we make, and it has never been a problem in the past.
<lifeless> stub: previously we did code + schema changes in lockstep, with several days (or more) qa on staging
<lifeless> stub: we're explicitly not doing that now...
<stub> Yes, and now we need to run the test suite with both /stable and /db-devel
<stub> Or we are going to have to start naming new columns with UUIDs
<lifeless> stub: well, its a risk assessment.
<stub> Yes, and next time we denormalize further we are going to call the column spn2?
<lifeless> stub: I guess I'm saying I'm fine with a name that reduces the risk; if you want a more risky but more consistent name, I can live with that
<lifeless> stub: I'd hope that we don't use spn unqualified, having learnt that lesson
<StevenK> stub, lifeless: So, I will kill this ec2 run, and I will investigate fixing the queries that use the full name of SPN and BPN unqualified.
<stub> I want consistent names. If we want to switch to spn/bpn  I'd like it to be consistent. This also gets tied up with migrating to text strings over integer ids
<lifeless> stub: such a migration might be a pessimisation for some queries (absent trigrams...)
<stub> Yes. We don't know yet.
<lifeless> stub: I think we need more investigation on that particular thing (though I know I was one of the proponents of it before :P )
<stub> For now, I'd rather stick with the full names and do two ec2 test runs (-b launchpad=stable, -b launchpad=db-devel)
<lifeless> stub: given stable is merged to db-devel all the time, just db-devel should be robust
<StevenK> stub: I know the test failures for db-devel anyway, so I will investigate those tomorrow.
<stub> True. So this isn't any different from before then?
<lifeless> stub: remember that someone could land a break in devel at any time; we probably need a deploy rule 'can only deploy a schema change if the current *live* revno has been tested by buildbot on db-devel'
<lifeless> stub: we'll figure out all the corner cases soon :P
<lifeless> matsubara-afk: are you around ?
<stub> Yes. We knew there would be extra landing complexity and more branches for allowing live patches. People thought the trade off was worth it.
<matsubara> lifeless, hey, yes. I'm in the QA sprint
<lifeless> stub: yup, and I still do :P
<stub> Which reminds me, I have branches to land since I couldn't get my branch to push last night due to major sucky 'Net.
<lifeless> matsubara: I want to patch oops-tools. It wants its own db cluster - is that still correct ?
<matsubara> lifeless, what do you mean by it wants its own db cluster?
<lifeless> matsubara: are there any docs on hacking it ? [the README.txt only talks about running locally...]
<matsubara> lifeless, hmm README.txt is the closest I have for docs.
<stub> StevenK: So you need to do what I was doing yesterday - separate branch from stable containing the required test fixes to land on devel, which gets merged into your db-devel so you can run the tests to ensure the required fixes really are the required fixes. And don't use devel for the source of your fixes or you risk leaking untested changes from devel -> db-devel.
<matsubara> lifeless, are you not able to run it locally to test your changes?
<lifeless> matsubara: I don't know how to run its test suite ;)
<matsubara> lifeless, make check
<lifeless> let me have another go at it :)
<lifeless> matsubara: I want to get rid of the custom oops parser
<matsubara> lifeless, right. basically you'll have to add your package to the download-cache, update versions.cfg and setup.py with the new package, remove the code from src/oopstools/oops/models.py and update it to use the code in the package. Do you need help with this?
<lifeless> matsubara: hopefully not :P
<nigelb> wallyworld_: my MP now works. I think I'll spend tonight / tomorrow morning writing tests.  Is there any more clean up that you'd suggest?
* gmb changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: gmb | Critical bugs: 238 - 0:[#######=]:256
<henninge> jml: Hi!
<StevenK> Dear Thunderbird, why are you taking up 1.2 cores?
<henninge> StevenK: you will have to email that question, not put it on irc ...
<StevenK> Bwaha
<lifeless> gmb: oh hai
<gmb> lifeless: Wotcher
<lifeless> can haz review?
<lifeless> https://code.launchpad.net/~lifeless/python-oops/extraction/+merge/71644
<lifeless> and
<lifeless> https://code.launchpad.net/~lifeless/launchpad/useoops/+merge/71634
<gmb> lifeless: Sure. I'll attend to those presently.
 * gmb grabs a cup of tea before getting down to work.
<lifeless> gmb: thanks!
<gmb> np
<Ursinha> hey guys :)
<Ursinha> I have a team subscribed to several packages, is it possible to mute bugmail in only one of them?
<Daviey> Ursinha: yes
<Ursinha> Daviey: using the ui?
<Daviey> Ursinha: It might be team admins only.. but for example, https://bugs.launchpad.net/ubuntu/+source/euca2ools/+subscriptions
<Daviey> I can edit the, "Ubuntu Server Team subscription" listed there
<Ursinha> Daviey: pedro was trying to edit the subscription of desktop-bugs of a package but there was no mute option
<lifeless> matsubara: so updated all the bits, ran make check:
<lifeless> bin/test
<lifeless> Creating test database for alias 'default'...
<lifeless> Traceback (most recent call last):
<lifeless>   File "/home/robertc/source/launchpad/oops-tools/parts/django/django/db/backends/postgresql_psycopg2/base.py", line 140, in _cursor
<lifeless>     self.connection = Database.connect(**conn_params)
<lifeless> psycopg2.OperationalError: could not connect to server: No such file or directory
<lifeless>         Is the server running locally and accepting
<lifeless>         connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5433"?
<matsubara> lifeless, does `psql -p 5433 lpoops` works for you? it looks like your postgres db for oops tools is not there
<lifeless> matsubara: thats what I was asking about needing a special cluster ! :)
<lifeless> matsubara: so the answer is 'yes, it needs a special cluster'
<danilos> Ursinha, I believe muting is person-only, teams can't have mutes (i.e. since you can't stop bug mail without leaving the team, we introduced muting; if teams don't want bug mail for a certain package, they should unsubscribe basically)
<Ursinha> right, doesn't apply for ubuntu use case :/
<poolie> jelmer, re your mps for lptools
<poolie> i think it's better to bring them in than to quibble about exactly how they're written
<poolie> so, 3x +1
<jelmer> poolie: gracias
<lifeless> jelmer: btw that loggerhead open-branch thing is failing 1000x a day
<jelmer> lifeless, I'm working on fixing it as we speak
<lifeless> jelmer: so if you're going to do other stuff first, let me know so I can roll it back
<lifeless> jelmer: ah cool
<matsubara> lifeless, :-)
<lifeless> matsubara: the README.txt tells how t make a pg 8.3 cluster
<lifeless> matsubara: does that need to be updated ?
<matsubara> lifeless, it should work the same for 8.4. if you can update: s/8.3/8.4/, I appreciate
<lifeless> matsubara: that seems happier
<allenap> jelmer: It looks like the prereq in https://code.launchpad.net/~jelmer/launchpad/bzr-code-imports/+merge/71625 might be out of date; the diff is >3000 lines suddenly.
<jelmer> allenap: Whoops, thanks - I'll set it back to wip for the moment while I get these other things sorted.
<gmb> jelmer: I just approved https://code.launchpad.net/~jelmer/launchpad/bzr-2.4b4/+merge/71624. Want me to run it through EC2 for you?
 * gmb does so anyway
<jelmer> gmb: Thanks, though my current branch will actually conflict with it (and has higher priority)
<gmb> jelmer: Oh, okay.
<jelmer> gmb: Thanks for the review though :)
<gmb> np :)
<lifeless> gmb: thanks; sorry about the size
<gmb> lifeless: No worries. After years of reviewing Celso's branches, 1500 lines is nothing.
<lifeless> gmb: and all fairly focused :)
<gmb> Indeed
<tumbleweed> query: archive.getPermissionsForPerson returns an empty list when connected anonymously. Is there any good reason for this? (I can't find a related bug)
<lifeless> probably means yiou have no permission to read that persons details anonymously
<tumbleweed> are such details not provided to non-ubuntu-developers?
<tumbleweed> for per-component permissions, it's pretty obvious from team membership
<lifeless> ubuntu has nothing to do with it
<lifeless> if you're getting an empty list the api probably got a list with inacessible entries in it
<lifeless> which it will filter
<lifeless> so try logging in :)
<tumbleweed> what I'm getting at is why does one need to be authenticated to read these
<lifeless> thats a good question
<lifeless> why shouldn't you be authenticated though ?
<tumbleweed> because this is for ubuntu-sponsoring. There is no good reason to be authenticated
<tumbleweed> (except this)
<poolie>  what does "expired token" mean?
<poolie> that the actual login has expired, or something else?
<lifeless> oauth tokens can be time limited
<lifeless> e.g. to a month o fuse
<lifeless> you need to request a new one when that happens
<poolie> gar
<poolie>         # TODO: handle the case of having credentials that have expired etc
<poolie>  
<tumbleweed> lifeless: would you like a bug report? its easily worked-around, but very non-obvious to the API client
<adeuring> lifeless: I sent an email about a batching issue (too long for IRC) to the lp-dev mailing list. could you have a look at itÃ
<adeuring> ...it?
<lifeless> Ã sure
<lifeless> tumbleweed: up to you, but I'm not convinced its a bug
<tumbleweed> lifeless: what is it then? It seems wrong to have to authenticate to read public information
<tumbleweed> lifeless: and having to authenticate cronjobs is an administrative pain
<tumbleweed> (IIRC this isn't the first time we've tried to do this, but we didn't remember the issue)
<lifeless> tumbleweed: we have restrictions on visibility on a lot of data
<lifeless> I'm not saying there is a good reason for this particular case
<tumbleweed> I have notice that I can use isSourceUploadAllowed (packageset authorisation), unauthenticated. But I can't determine PPU or component-based upload rights
<lifeless> anyhow, its much better for us if cronjobs are authenticated
<lifeless> because when they go rogue and use a tonne of resources, its a pain to track anonymous sessions down
<tumbleweed> fair enough
<tumbleweed> this one is quite a beast (~10 min running time)
<wgrant> Ursinha: Why can't the team unsubscribe?
<poolie> oh ffs
<poolie> any http error maps into keyerror
<Ursinha> wgrant: because we used the list of packages a team is subscribed to to say which packages should the team care about
<Ursinha> *we use
<jml> henninge: hi
<poolie> which is bug 626960
<_mup_> Bug #626960: Collection dictionary access should not return KeyError when the server is down <lazr.restfulclient:Triaged> < https://launchpad.net/bugs/626960 >
<henninge> jml: Hi! I resolved it already.
<henninge> jml: testtools' setup.py does not know egg_info
<henninge> and does not need it
<jml> indeed it shouldn't :)
<henninge> I am quite new to setuptools, so I am not sure why
<henninge> but sdist produced the right package
<poolie> o/ jml
<jml> poolie: hi
<wgrant> Ursinha: Ah :(
<Ursinha> wgrant: so we're workarounding it by creating teams only to be collections of packages, let's say
<wgrant> You could also just keep a list of names around :)
<wgrant> But yeah, using the subscription list is handy.
<Ursinha> wgrant: haha no way...
<wgrant> I have a few teams that I do it for.
<wgrant> Much easier to manage.
<Ursinha> yet another list of things... better keep in launchpad
<wgrant> But unfortunate for notifications, indeed.
<poolie> hello gmb
<poolie> could you review https://code.launchpad.net/~mbp/lazr.restfulclient/626960-keyerror/+merge/71667 please
<poolie> and perhaps land https://code.launchpad.net/~james-w/lazr.restfulclient/reject-non-json-responses/+merge/16960
<gmb> poolie: Sure, I'll take a  look shortly.
<poolie> thanks matey
<jtv> danilos: did you just make qastaging send me email?  I thought that was supposed to be captured?
<jelmer> poolie: can you land my lptools branches or should I apply for team membership?
<lifeless> matsubara: so, have tests running, I will fiddle with this tomorrow more
<matsubara> lifeless, cool.
<jelmer> is there a publically accessible list of things that are in a specific package set?
<StevenK> jelmer: You can query that sort of thing over the API
<jelmer> StevenK: close enough - thanks
<danilos> jtv, hum, interesting, I did some QA this morning for subscribing large teams to PPAs, but qastaging shouldn't send any emails
<jtv> That's what I thought.
<jtv> And then I got that email about your privates.
<jtv> About your empty privates, in fact.  Which, on reflection, may not have been your first choice for a message to be sent out to the world.
<danilos> jtv, empty privates, though
<danilos> :)
<danilos> jtv, actually, it sucks that the emails have gone through (all of ~canonical would have gotten it)
<jtv> We can only hope that qastaging email is at least whitelisted.
<jtv> As they were at the time.
<lifeless> I didn't see any such mail
<danilos> jtv, I didn't get anything, so I hope we are all good
<jtv> I got two.
<jtv> But then I think I'm whitelisted for staging email.
<danilos> jtv, right, that explains it then
<jtv> I hope it does!
<jtv> Wow.  Still hearing a thunderclap that started about 20 seconds ago.
<jtv> The lightning was pretty far off (5km at least) but shockingly bright.
<danilos> lifeless, hi, do you perhaps know what's our limit for storm caches? I am trying some load_referencing and similar magic, and it seems I am hitting the limit because depending on the order, different statements show up
<lifeless> I don't, no. I am rather of the opinion that we need to set it as high as it can be without thrashing, because otherwise it just forces pathological behaviour - and we wipe the cache between requests anyway.
<lifeless> danilos: there is a policy knob in the lazr config that controls it, IIRC
<lifeless> all that said, every time I've looked at similar symptoms, we haven't had a cache limit problem, rather the cache working on the 2nd page in a test
<lifeless> or similar
<lifeless> (because test view instantiation doesn't reset the cache - the storm resetting only happens in thread 1+, not 0+
<nigelb> 56
<danilos> lifeless, right, thanks, I'll try to dive deeper (I am using ++profile++show to look at the queries being issued)
<lifeless> danilos: qastaging, or dev?
<danilos> lifeless, dev
<lifeless> don't forget celebrities do get cached
<lifeless> hmmm, way past bedtime. I have to crash()
<lifeless> gnight all
<danilos> night-night, lifeless
<deryck> Morning, everyone.
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #977: FIXED in 5 hr 46 min: https://lpci.wedontsleep.org/job/devel/977/
<bigjools> 5 hr 46 min!
<jtv> gmb: got reviews for you if you've got time.  You'll enjoy this one: https://code.launchpad.net/~jtv/launchpad/more-820456/+merge/71689
<gmb> jtv: Just nomming at the moment. Queue 'em up, though and I'll come to them when I return.
<jtv> gmb: there's one that may not jump out at you from the queueâbrad started on it once but never had time to finish it.
<jtv> So it's claimed, but actually looking for a new reviewer.
<jtv> Bon appÃ©tit.
<gmb> jtv: okay. Can you add me as a reviewer so I don't lose track of it?
<jtv> Aye-aye.  Thanks.
<gmb> Cool, and np
<deryck> abentley, ping.  I can chat whenever you like now.
<abentley> deryck: I'll just get coffee...
<deryck> abentley, hmmm, me too :)
<deryck> What's the old qa report page we used to cut-n-paste bug numbers from when asking for a nodowntime deploy?
<deryck> matsubara, can you point me at it? ^^
<matsubara> deryck, You mean the report that used to be on lpqateam.devpad.com? If yes, see: https://bugs.launchpad.net/lp-devops-dashboard/+bug/823886
<_mup_> Bug #823886: New dashboard design removed available deployable revision and fixed bugs list <Launchpad DevOps Dashboard:Triaged> < https://launchpad.net/bugs/823886 >
<deryck> matsubara, yeah, I was looking for that list.  Didn't know it didn't exist anymore.
<matsubara> sorry about that. I'll get to it once I get back from the QA sprint. Meanwhile I guess the only way to get that info is to see the deployment report itself
<deryck> matsubara, gotchas.  no worries, thanks!
<abentley> deryck: actually, wallyworld has landed a rollback as 13696.
<deryck> abentley, ah, nice!  So we should be deploy ready then, once the qa pages catch up?
<abentley> deryck: Looks like.
<deryck> abentley, awesome.
<abentley> matsubara: buildbot approved revision  13697 an hour ago, but it's not included on https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html Do you know why?
<matsubara> abentley, will look into it in a moment
<abentley> matsubara: thanks.
<jelmer> gmb: can I add another MP to your queue?
<nigelb> bigjools: I haz success \o/
<nigelb> Well, not entirely, but mostly :)
<bigjools> nigelb: congrats
<nigelb> The MP looks small now, but I need ta add tests :)
<gmb> jelmer: Sure.
<jelmer> gmb: Great :) The MP is https://code.launchpad.net/~jelmer/launchpad/safe-opener-threading/+merge/71670
<gmb> k
<stub> Who is the owner of imported bug comments?
<gmb> stub: The Person who made the comment (we create one if they don't exist in LP)
<stub> k. not a robot. good.
<matsubara> abentley, r13697 is in the deployment report now
<abentley> matsubara: thanks.
<matsubara> np
<abentley> deryck: The board is green for a deployment.  Can you talk me through it?
<jtv> thanks for the review gmb
<deryck> abentley, yay!  Indeed I can.
<gmb> jtv: np.
<deryck> abentley, looking for links....
<deryck> abentley, so it's really only a matter of filling out the table under "Requested stable Deployments" on the LaunchpadProductionStatus wiki page....
<deryck> abentley, you have to paste all the bug numbers fixed with links in that table.  but the revno is the highest revno to deploy.
<deryck> abentley, approved by is "qa-tagger" and deploy to is "nodowntime"
<deryck> abentley, and you can see example just above that have been deployed.  then you ping a losa once the wiki page is filled out.
<abentley> Okay.
<deryck> abentley, it takes around 2 hours to do the deploy and you want to be on hand for that time.
<deryck> I think that's it.
<jelmer> abentley: great to see that fix for bug 386596, that also helps towards build-from-branch-into-primary.
<_mup_> Bug #386596: pushing to a packaging branch can't create a new package <codehosting-ssh> <escalated> <lp-code> <not-pie-critical> <package-branches> <principia> <qa-ok> <stakeholder> <Launchpad itself:Fix Committed by abentley> < https://launchpad.net/bugs/386596 >
<abentley> losa: please deploy 13697.  I have updated LPS.
<abentley> jelmer: cool.
<allenap> gmb: Fancy a short one? https://code.launchpad.net/~allenap/launchpad/wrong-needs-linking-count-bug-795359/+merge/71715
<gmb> allenap: I'll add it to the queue.
<allenap> gmb: Ta :)
<gmb> np
<jtv> thanks again gmb!  And I just put up another one.
<gmb> jtv: Hah. Might just be able to get that one done before my head melts.
<gmb> :)
<jtv> gmb: I'll stop here then.  :-)
<gmb> Performance Tuesday | https://dev.launchpad.net/ | On call reviewer^W^W^Wjtv's review monkey: gmb | Critical bugs: 238 - 0:[#######=]:256
 * gmb is glad he forgot to put /topic in there, actually :)
<jtv> cjwatson: did you get my email about custom-uploads copying?  Unless you see any problems with the design, it's all set to go in at your word.
<allenap> Thanks gmb :)
<gmb> allenap: np :)
* gmb changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer:  - | Critical bugs: 238 - 0:[#######=]:256
<gmb> jtv: approved.
<jtv> gmb: thanks, and feel free to take the rest of the day off.  :)
<gmb> Ta :)
<mtaylor> lifeless: if I'm consuming launchpad openid auth, are timeouts your end, my end, or just the way life is?
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #813: FIXED in 5 hr 48 min: https://lpci.wedontsleep.org/job/db-devel/813/
<abentley> deryck: there's no OCR, so could you please review https://code.launchpad.net/~abentley/launchpad/fix-unicode-path/+merge/71762 ?
<deryck> abentley, yeah, definitely.
<abentley> deryck: thanks.
<deryck> np
<lifeless> mtaylor: +login ?
<mtaylor> lifeless: yeah?
<mtaylor> lifeless: (basically, I keep having to re-auth to launchpad all the time in my jenkins, and I was wondering if that was a config thing on my end, or just the way life goes)
<lifeless> oh, you don't mean 'the login process times out', you mean 'the tokens seem to run out of life' ?
<abentley> lifeless: are you around when the LOSAs BOD?
<lifeless> BOD ?
<lifeless> oh, start.
<lifeless> I can be.
<abentley> lifeless: We've done a no-downtime deploy of the push-creates-source-package, but they didn't have time for the codehosting deploy.
<abentley> lifeless: I can handle it tomorrow if it's not convenient for you.
<lifeless> abentley: the losas want stuff RT'd while they are sprinting; can you: put it on LPS as a request, RT it (and cc: me on the mail to RT so I get copied), and if I'm around I will touch base with the first losa to show up and see about doing it
<abentley> lifeless: Okay.
<lifeless> thanks
<abentley> lifeless: can I use you for "Approved by"?
<lifeless> yes
<deryck> abentley, r=me on the review.  Looks good.
<abentley> deryck: thanks.
<deryck> abentley, np!
<sinzui> jcsackett, do you have time to mumble?
<jcsackett> sinzui: 15 min?
<sinzui> sure
<sinzui> deryck, bug 173019 may not be as easy as you think and I doubt it is high given the age of the bug and the lack of duplicates
<_mup_> Bug #173019: "Renewal period" is marked as "(Optional)" when it isn't <lp-registry> <teams> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/173019 >
<deryck> sinzui, hey.  I thought I marked that low actually.
<sinzui> :)
<deryck> maybe my fingers didn't do what my brain did :-)
<sinzui> deryck, Past discussion was to use a multistep form, or an overlay to only ask for information that is needed
<deryck> sinzui, and I just thought "easy" because I assumed we could drop the "optional" label.  If that's not possible without a larger fix, forgive my lack of knowledge.
<sinzui> deryck, understood. We do not know how to express invariants as a state between optional and required
<deryck> sinzui, ah, gotchas.
<jcsackett> sinzui: i'm on mumble whenever you're free.
<lifeless> can we delete shipit.css now ?
<bac> lifeless: could i get your thoughts on https://code.launchpad.net/~bac/launchpad/bug-813322-2/+merge/71733?  not asking for a code review, just feedback on the approach.
<lifeless> sure
<lifeless> looking
<lifeless> sweet
<lifeless> bac: looks fine; I think that we will find we need to another pass for backend efficiency (e.g. perhaps a separate xaction for each dup we move across, but thats a different ratchet - we're not at that point in our optimisation arc yet.
<lifeless> bac: so a big +1 from me.
<bac> lifeless: great, thanks
<lifeless> I need brainstorming help
<lifeless> we have page ids
<lifeless> which are kindof a unique part-of-the-codebase-to-look-at
<lifeless> cronscripts are also unique-parts-of-the-codebase-to-look-at
<lifeless> but putting 'sendbranchmail' in as a pageid rubs me the wrong way.
<lifeless> I want to find a generic term that I can use in the public contract for oopses, and migrate towards for the serialized form as well
<benji> "task-id", "action-id", "activity-id", "functionality-something"
<lifeless> good, good, keep em coming!
<lifeless> another one I thought of was 'context'
<benji> I like "facet" but we already use that term in LP
<lifeless> thats ok
<lifeless> this is for python-opos
<lifeless> python-oops
<lifeless> which LP will use
<lifeless> but needs to be suitable for u1, landscape, possibly isd, possibly ubuntu, package-importer
<lifeless> its kind of like user-agent
<lifeless> 'agent' ?
<lifeless> like, agent: zope3/Person:+index
<benji> "concern"
<lifeless> thats good
<benji> "agent" is very close to "client" in my head, so that seems a little confusing
<lifeless> I see
<benji> I don't really like it, but "relm" comes to mind.
<lifeless> realm ?
<lifeless> thats another possibility
<benji> heh, yeah I typoed the "a"
<lifeless> if you saw : "Lucid/apport;program=evolution" , would that make sense to you ?
<lifeless> or would : "evolution" with other fields noting it was from a lucid box be better?
<lifeless> I'm thinking about how to write up a recommended use
<benji> to me, "Lucid/apport;program=evolution" communicates well (I think, here's what I think it means: "this error was reported by apport on Lucid but the app generating the error was evolution")
<lifeless> bac: yeah
<benji> you might break those into two fields, the reporter (which might well be an "agent") and the topic being reported ("program: evolution")
<lifeless> bah
<lifeless> benji: yeah
<lifeless> topic and reporter, I like.
 * lifeless things
 * lifeless thinks.
<lifeless> so the difference between pageid and url
<lifeless> is that one pageid applies to many urls.
<lifeless> theres one other concept we might want, which is for multitenancy
<lifeless> which I think reporter really fulfils
<lifeless> claiming a reporter of launchpad gets you into the launchpad oops bucket that only lp devs can see
<lifeless> folk can inject crap in there if we have an anonymous submission thing, but we can filter that out programattically.
<benji> I wonder if some loose hierarchy would work.  Something like "ubuntu/lucid/appport" and "canonical/launchpad".
<benji> Although, you can get in several forms of one-size-fits-all trouble with that approach.
<lifeless> so reporter: 'Describes the program reporting the oops. For instance you might put your program name in here, or your website domain.'
<lifeless> topic: 'The subject or context for the oops. For instance with a command line program you might put the subcommand in here, in a website you might put the template or function name (something that is common to many different urls)'
<lifeless> apport would do 'reporter: lucid/apport' topic:'evolution'
<lifeless> gpverify would do 'reporter: launchpad/gpgverify' or just 'reporter:gpgverify' and for topic the api being called.
<benji> why would apport include "lucid"?  I would think that'd be included in some other field if needed (along with all kinds of other environmental info).
<lifeless> our main appservers would do 'reporter: launchpad/main' or just 'launchpad' and for topic what we currently put in pageid
<lifeless> benji: oh, so one of the big things ubuntu have is that cross-release crashes are generally considered irrelevant
<lifeless> benji: that would just be an easy way to trigger partitioning of the analysis
<benji> seems reasonable, "lucid" really is intrinsic to the identity of the reporter then
<lifeless> I like this - thanks for the brainstorm
 * lifeless goes to make it realityish
<benji> yep, any time
<mtaylor> lifeless: yes - I mean "the tokens seem to run out of life"
<lifeless> mtaylor: I don't believe we have a mandatory expiry.
<lifeless> mtaylor: what expiry are you choosing when you create the tokens?
<mtaylor> lifeless: ok- so that's more likely something on my side then
<mtaylor> lifeless: not sure - let me poke around (it's in java, so it takes a moment) :)
<lifeless> back in a ninute
<lifeless> back
<lifeless> mtaylor: ^
<lifeless> sinzui: I had a wtf moment when I saw your branch name 'private-bug-1'
<sinzui> lifeless, I think I could make that bug private without a brach
<lifeless> sinzui: I think yo ucould too
<lifeless> sinzui: still, I had a moment :)
<sinzui> We could retitle the bug to be Microsoft has a dominate share of the increasing irrelevant desktop market
<lifeless> :)
<lifeless> well
<lifeless> we could try.
<lifeless> Who knows, it might not timeout.
<sinzui> yeah. We probable cannot close the bug without making Lp twice as fast
<StevenK> sinzui: http://irclogs.ubuntu.com/2011/08/10/%23launchpad-dev.html
#launchpad-dev 2011-08-17
<nigelb> wallyworld_: \o/
<nigelb> It works, except for the anonymous user case, which I'm fix0ring now :)
<lifeless> can has review? https://code.launchpad.net/~lifeless/python-oops/extraction/+merge/71794
<lifeless> can has review two? https://code.launchpad.net/~lifeless/python-oops-datedir-repo/extraction/+merge/71795
<lifeless> can has review three? https://code.launchpad.net/~lifeless/launchpad/useoops/+merge/71796
<lifeless> was there any objection to just nuking 2011-08-17 01:12:19 WARNING Found 2 competing templates with translation domain 'javascript': "openerp-javascript in OpenERP Web Client 6.0"; "view-calendar-javascript in OpenERP Web Client 6.0".
<lifeless>  ?
<wgrant> lifeless: Why has that started spamming us now, anyway?
<wgrant> lifeless: I wonder if somebody broke asuka's mail capturing.
<lifeless> probably puppetisation
<lifeless> rt 47413
<lifeless> and set to prior 90
<wgrant> Would be nice if we were warned when nasty machines like that were puppetised, so we could watch for breakage.
<wgrant> Only 90?
<wgrant> We may be spamming innocents.
<lifeless> 90 is zomg
<wgrant> All Zopeless mail is probably untrapped.
<lifeless> drop-everything
<wgrant> Ah
<lifeless> the only higher thing I can sanely do is ring the escalation phone at 2:25am
<lifeless> This hasn't been picked up on for several days.
<wgrant> Yeah.
<lifeless> so I think that would be overkill
<wgrant> Indeed.
 * lifeless looks around for a revuer
<StevenK> lifeless: Hm?
<lifeless> StevenK: oh hai; I have three small reviews up ^
<lifeless> StevenK: (they really are small. The Genuine Article.)
<jtv> StevenK, wgrant: mind if I update & restart dogfood?
<StevenK> jtv: Go ahead
<jtv> OK
<lifeless> StevenK: thanks!
 * StevenK tries to get his eyes unglazed after reading 4,000 lines of diffs
<lifeless> time for a vodka tonic ?
<StevenK> At midday?
<lifeless> every 4000 lines :P
<lifeless> code review drinking game
<StevenK> Oh, I was going to go with "Every time you mark an MP as approve, take a drink. Every time you mark an MP as Needs Fixing, finish the glass."
<lifeless> ah, thank you internet: http://movieboozer.com/2011/07/04/source-code-2011-drinking-game/ close enough
<lifeless> StevenK: that works too
<lifeless> fun
<lifeless> http://www.depesz.com/index.php/2011/07/06/bloat-happens/#more-2231
<jtv> btw StevenK, wgrant: I put a script on dogfood that updates the code / dependencies / db permissions, rebuilds the current branch, and restarts the app server.
<jtv> Takes a few minutes but it's convenient, and can save us the "damn this time a partial update doesn't quite do it" exceptions.
<jtv> *Unfortunately* it breaks at the "make" where the Makefile tries to rm -rf /var/tmp/mailman, which is root-owned on dogfood.
<wgrant> A few minutes?
<wgrant> You mean a few hours?
<jtv> Althoughâ¦ no, /var/tmp/mailman's been fixed.
<jtv> wgrant: no, a few minutes.
<jtv> It doesn't update the database (though it does update the schema)
<jtv> Of course if you didn't run it for months on end, it'd probably take a lot longer.  But having it be easy to run would also solve that.  :
<jtv> :)
<jtv> I just ran the thing, actually.
<jtv> wgrant: have a look â ~launchpad/bin/upgrade-dogfood-launchpad
<wgrant> I've only SSHed into the DC once this week, and I'm trying to not do it again :)
<jtv> OK OK OK
<jtv> Just try it sometime.  :)
<StevenK> Haha
<jtv> Uh-oh.  StevenK's laughing at me.  This can't be good.
<nigelb> wgrant: Are you still on vacation?
<jtv> nigelb: if you have to ask, I guess that means he never was.
<nigelb> jtv: haha. I think he needs to vacation in some small island without internet.
<jtv> I thought he lived on one?
<nigelb> "small"
<jtv> It's that blip just off the cost of Asia, isn't it?  It has that airport where you refuel on the way to South America.
<StevenK> jtv: You can talk
<jtv> Yes.
<nigelb> haha
<jtv> Yes, I can.
<StevenK> jtv: I think Australia has more Internet than Thailand
<jtv> That can't be right.  Thailand has more internet than any other country in the world.
<jtv> Measured by the number of sites the government censored, at least.
<nigelb> jtv: is in Thailand?
<jtv> More than China.  Come on, who's laughing now?
<nigelb> I thought it was stub in Thailand.
<jtv> Yes, yes, you're right.  I'm mistaken.
<lifeless> nigelb: we are allowed > 1 person in a country :)
<lifeless> jamesh: hi
<jamesh> hi lifeless
<lifeless> jamesh: did you release that python-gpg update?
<nigelb> lifeless: haha
<jamesh> no I haven't.  I may as well sort that out now.
<lifeless> jamesh: I can put a nag into cron if thats better for you :P
<lifeless> jamesh: also how much do you know about wsgi-oops
<jamesh> lifeless: I've only made a few small changes to it.  It started out with the Landscape team trying to extract the LP OOPS system, and then we added some WSGI glue to that
<lifeless> jamesh: yeah
<jamesh> what did you want to know?
<lifeless> jamesh: I'm redoing that but differently... and migrating LP at the same time rather than never :)
<lifeless> jamesh: well, the traceback extract grabs an awful lot of data
<lifeless> jamesh: thats not marshallable in a sane way into the current rfc822 format
<lifeless> jamesh: but I'm wondering whether its been useful to folk; should I file a wishlist bug on python-oops to port that code across ?
<jamesh> lifeless: back when I was on the LP team, one of the long term goals was to switch to storing OOPS reports as zip files, as a way to make it easy to represent attachments like the traceback and sql statement log.  Obviously that has never happened
<jamesh> lifeless: at this point, if the reports are readable by oops-tools, I don't think many people would care what format they are stored in
<lifeless> jamesh: right; I'm more talking about the stack-walking variable extraction code than format
<jamesh> oh.
<lifeless> jamesh: porting oops-tools to use lp:python-oops-datedir-repo/serializer_rfc822 is fairly straight forward - and once thats done migration to e.g. bson dicts is trivial.
<jamesh> lifeless: are you talking about wsgi-oops, or the LP oops code here?
<lifeless> jamesh: wsgi-oops has stack-variable-extraction, LP oops doesn't (it just uses zope.exceptions.format_exception), oops (the new module) doesn't, (It just uses traceback.format_traceback)
<jamesh> I remember the LP oops code used zope's traceback collection API, but the wsgi-oops one seems fairly simple
<jamesh> oh.  I see what you're talking about.
<jamesh> I don't think the locals or globals dictionaries from the traceback frames ever make it into the serialised oops report
<lifeless> jamesh: http://bazaar.launchpad.net/~canonical-launchpad-branches/python-oops/trunk/view/head:/oops/createhooks.py#L91
<lifeless> jamesh: right, because the rfc822 format is so limited ;)
<jamesh> I know the LP code used Zope's traceback formatter, which did include some locals in the output
<lifeless> jamesh: yeah, so the lp errorlog module now installs its own exc_info grabber
<lifeless> which is about the same as the code I just linked to, but usin the zope thingy
<jamesh> lifeless: so, given that we aren't storing this data, perhaps it would be okay to leave it out of the new system you're building.  If it is useful, then perhaps we can work out a way of storing it later.
<lifeless> jamesh: thanks
<lifeless> jamesh: I had tentatively concluded that
<lifeless> time to register yet another lp project
<lifeless> python-oops-wsgi, to avoid confusion :P
<jamesh> lifeless: it might be worth checking up with the Landscape folks to see if they are using a different serialiser that does actually store the data though.
<jamesh> (or you might have already done that)
<lifeless> jamesh: I've chatted with them about this project, not about that specific bit of detail.
<lifeless> it should be easy enough to port the logic later
<lifeless> my first goal is a generic, used by LP, and used by a non-LP wsgi app (the gpgverify one by choice), implementation
<lifeless> with strict dependency boundaries - for folk shipping it, don't want to bring in the universe
<lifeless> second goal will be updating oops-tools to use the parser module from datedir-repo to replace its homegrown one
<lifeless> then we can add a new serialiser to datedir-repo and win.
<jamesh> lifeless: https://launchpad.net/pygpgme/trunk/0.2
<lifeless> jamesh: sweet
<lifeless> now to get it uploaded $everywhere :)
<lifeless> jamesh: http://pypi.python.org/pypi/pygpgme
<jamesh> I'm doing that.
<lifeless> jamesh: heh, sorry!
<jamesh> http://pypi.python.org/pypi/pygpgme/0.2
<lifeless> jamesh: awesome
<lifeless> I'll chase up lp's dep versions a little layer
<poolie> lifeless: hi; how would we go about getting a release of restfulclient and co
 * poolie is hoping for something better than 'do it myself'
<lifeless> https://dev.launchpad.net/ReleaseChecklist
<lifeless> poolie: it is do it yourself
<poolie> lifeless: what i really meant is, can't the lp core team do it?
<lifeless> modulo the pypi permissions (we -really- would benefit from a teams module there)
<lifeless> and I can grant pypi permissions as needed, I think flacoste added me to everything
<lifeless> this checklist needs trimming and automation, rather desperately, but it is functional
<poolie> phut
<poolie> i feel like i'm already kind of doing at least half the work by fixing the bug
<poolie> oh well
<lifeless> sure
<lifeless> I mean, thats what bzr contributors expect - they fix, core team releases on $whatever schedule
<lifeless> are you asking because a release is long overdue? or its particularly urgent?
<poolie> the first
<poolie> let me check if this is actually true though
<poolie> not all that long
<poolie> perhaps the difference is there's no obvious schedule
<lifeless> how much merged work is sitting unreleased?
<poolie> though, that's also somewhat true for bzr plugins
<poolie> just two https://launchpad.net/lazr.restfulclient/+milestone/0.11.3
<poolie> it's not urgent, it just would be a shame to have them sit for a long time
<poolie> i thought there were more in other lp dependencies
<lifeless> so one possibility is that the system is working
<lifeless> another possibility is that with leonard gone, folk are not releasing-on-land, nor will anyone release-on-schedule
<jtv> \o/ I think I've figured out what's breaking rocketfuel-get on dogfood.  When it looks for "sourcecode" dirs to re-link, in dogfood's setup it also finds lp-sourcedeps.  Maybe rocketfuel-get should exclude lp-sourcedeps explicitly.
<poolie> that's the uncertainty that was in my mind
<lifeless> I think some discussion on list is in order
<lifeless> I'm not sure what (or even whether) we have a policy about doing-releases-of-things-we-garden [and perhaps we don't need one... but perhaps we do]
<lifeless> jtv: isn't lp-sourcedeps normally a level up ?
<jtv> Normally, yes.
<jtv> But these scripts evolve over time.
<jtv> So it depends on the age of the LP installation.
<jtv> And in this case, permissions and responsibilities.
<jtv> If my hunch is right, fixing up rocketfuel-get is the least invasive.
<lifeless> ugh
<lifeless> failing buildmaster tests
<lifeless> could not stab /var/tmp/buildd/build-slave.pid
<StevenK> Oh, screw you, testrepository
<lifeless> StevenK: ?
<lifeless> oh, and for bonus I have a test fail in there too
<StevenK> testr failing --list => no output, testr run --failing => I'll just run every single test, lol
<lifeless> StevenK: heh
<StevenK> With no output
<lifeless> StevenK: bit of a foot-gun event, I guess.
<StevenK> steven@liquified:~/launchpad/lp-branches/denorm-bspph% testr run --failing
<StevenK> running=xvfb-run ./bin/test --subunit
<lifeless> jamesh: do you happen to know how I can reset gnome-gpg to ask for my passphrase again ?!
<lifeless> and preferrably nuke the silly thing that ever captured it
<jamesh> lifeless: delete the passphrase from the keyring using the "Passwords  and Encryption Keys" app
<jamesh> I haven't used gnome-gpg in years (gnupg-agent seems to do a good job)
<lifeless> jamesh: that seems to want to change the actual gpg managed passphrase
<lifeless> jamesh: which isn't what I intend
<jamesh> lifeless: you're looking on the passwords tab, right?
<jamesh> not the "My Personal Keys" or "Other Keys" tabs
<lifeless> jamesh: it shows 'passwords: default and 'Passwords: login'
<jamesh> lifeless: right.  expand those and find the entry gnome-gpg added.
<lifeless> there isn't one :)
<jamesh> well, that's where gnome-gpg would be storing the passphrase
<lifeless> ah
<lifeless> I had to click 'unlock'
<lifeless> to unlock the first folder, before it would show me its contents.
<lifeless> ui win!
<lifeless> jamesh: thanks!
<jamesh> "passwords: default" would be a keyring from before the PAM integrated login keyring
<jamesh> if it doesn't hold anything interesting you would probably be best off deleting it.
<lifeless> jamesh: I've been running this homedir uninterrupted since, uhm, 2000
<lifeless> jamesh: it appears to have a lot of semi interesting things
<jamesh> the login keyring is locked and unlocked automatically by pam
<jamesh> the default one requires manual password entry (assuming it has a password set)
<lifeless> thanks
<lifeless> bbiab
<StevenK> jtv: O hai. Do you have time for a small review?
<jtv> Yes, okay
<StevenK> jtv: https://code.launchpad.net/~stevenk/launchpad/denorm-bspph-qualified/+merge/71812
<jtv> I find it odd that you should come to me with this request, after all my hard work to scare you off.
<StevenK> jtv: lifeless owed me, but he's buggered off. And you're supposed to be OCR anyway
<jtv> ah yes
<jtv> Anyway, I see a familiar pattern: it's as if someone has tried to make the SQL harder to figure out.
<StevenK> Hah
<jtv> By arbitrarily turning some keywords into lower-case, and putting the line breaks in odd places.
<jtv> Could you please just put the FROM, in all caps, at the beginning of the line?
<StevenK> jtv: In bugtask?
<jtv> Yes.  And maybe make the other SQL keywords consistently upper-case as well?
<wgrant> I wish it was practical to write a lint check to force that.
<jtv> Yes!  Oh God, yes!
<jtv> And go away.  You are not here.
<wgrant> Few reviewers do.
<wgrant> Hah.
<jtv> Reviewers nowadays are a bunch of pussies.
<jtv> "Oh yes, I can't read this, but if you get it past the python parser I'll approve it."
<StevenK> jtv: Anything else?
<jtv> AFAICT it's all cosmetic.  Am I missing something?
<StevenK> wgrant: I'd solve the bloody problem and storm-ify the methods, but they make my eyes bleed.
<StevenK> jtv: It's lead-up to adding a sourcepackagename column to SPPH
<lifeless> if you can make a lint check for it, you can make an automated fixer for it.
<jtv> Then go forth and land ye your branches, that they may be merged.
<jtv> lifeless: you can, but lately I've really come to appreciate the affirmative power of manual formatting: "yes I've thought about this, yes I mean this to be in the source code as it is."
<jtv> I even rely on it by not inserting proper spacing into temporary debug statements etc, as an extra safeguard against accidentally leaving them in.
<lifeless> mmm, I can see an argument for that, but I'm not convinced
<jtv> I'm not making an argument, I'm telling you how it is.  It helps.
<wgrant> Let's just version control the AST.
<lifeless> if python was just a little less fragile
<lifeless> now where, was I, thats right, housework. Then extracting code to oops-wsgi
 * jtv wonders if that is pronounced "oopski"
* jtv changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer:  jtv | Critical bugs: 238 - 0:[#######=]:256
<LPCIBot> Project db-devel build #814: FAILURE in 5 hr 47 min: https://lpci.wedontsleep.org/job/db-devel/814/
<jtv> thanks for the surprise review, lifeless :)
<lifeless> jtv: de nada
<rvba> StevenK: Hi! Thanks for the review Steven, I made the corrections you suggested.
<adeuring> good morning
<jtv> morning adeuring
<adeuring> hi jtv!
<jtv> StevenK: you're not still reviewing, are you?  Because I've got this: https://code.launchpad.net/~jtv/launchpad/bug-824553/+merge/71823
<bigjools> morning all
<mrevell> Good morning
<StevenK> rvba: r=me
<rvba> StevenK: Thanks!
<stub> lifeless: Was the reason pqm results weigh in at 400k for a single test failure because of the Librarian logs not being cycled between tests, or something else?
<lifeless> librarian
<lifeless> there is a bug open I think; all that needs to happen is the fixture glue to be tweaked to only attach new rows from the log, rather than all of it.
 * stub wonders how the Librarian would cope with that many HUPs
<lifeless> or truncate the log, or something.
<lifeless> stub: stress test time!
<stub> HUP HUP HUP HUP HUP OH JUST FSCK OFF
<lifeless> stub: one easy way would be to diff the log in memory - read or stat at the start, and etc
<stub> Maintain an open file, seeking to the end after each test.
<stub> fsync before the seek if you care.
<stub> somehow handle the tests that test Librarian log file rotation :-)
<stub> How good is the bzr cherry picking story now? If I merge a revision from db-stable into devel, will it cause a conflict when stable is merged to db-devel?
<poolie> hi all
<poolie> any lazr or lplib experts that could look at https://code.launchpad.net/~mbp/udd/819910-service-root/+merge/71832 ?
<poolie> is there a more kosher way to do it?
<lifeless> stub: it may; but if the texts are identical it should never.
<lifeless> and -woo- we have a wsgi layer.
<stub> So as long as I don't have to resolve conflicts should be fine.
<lifeless> and its (at least to me) a lot cleaner than the launchpad-loggerhead on. I may be biased.
<cjwatson> jtv: sorry, I know I still owe you an assessment of your custom upload plans, and I promise to figure out what I think today - but any chance you could review https://code.launchpad.net/~cjwatson/meta-lp-deps/fix-target-detection/+merge/71840 for me?  there was a regression when IS tried to deploy my previous meta-lp-deps change last night
<jtv> cjwatson: funnily enough I was just composing another IRC message for you.  :)
<jtv> I'm having a look.
<lifeless> poolie: whats your cli for pqm called again ?
<lifeless> (and bugs etc, your nih api client :P)
<jml> lifeless: hydrazine?
<lifeless> yes!
<jml> lifeless: not to be confused with lptools or ubuntu-dev-tools ("ubuntu-dev" being code for "launchpad" there)
<poolie> feed-pqm
<poolie> in hydrazine
<poolie> the nih api client is 'wrested'
<lifeless> poolie: I needed it to do that review for you :)
<poolie> which?
<lifeless> hydrazine
<jtv> cjwatson: taking a while for the MP to update.  Did you get a failure email by any chance?
<cjwatson> not yet ...
 * cjwatson pokes offlineimap just in case
<cjwatson> jtv: it seems to be up to date when I look from here
<jtv> cjwatson: yup, just got it at last.
<jtv> cjwatson: not much I can do with this one, but reviewed.
<jtv> Meanwhile, about those custom uploads...  :)
<jtv> cjwatson: can you land it yourself?  I'm not even sure I can, tbh
<jtv> Your MP I mean, not mine.
<danilos> jtv, heya, I've got a branch up for review: https://code.launchpad.net/~danilo/launchpad/bug-826692/+merge/71836
<jtv> danilos: do you now?  Good for you mate!
* danilos changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jtv, danilos | Critical bugs: 238 - 0:[#######=]:256
<cjwatson> jtv: I can't; it should just involve merging it into lp:meta-lp-deps, which is owned by ~launchpad-committers and which apparently you just push stuff to directly rather than via PQM
<danilos> jtv, yeah, I am just bragging around, thanks
<cjwatson> judging from the commit history
<jtv> cjwatson: yes, that's how it happens with the less well-managed branches.  Can't say I'm familiar with this one.
<cjwatson> jtv: I can deal with talking to IS about the actual deployment stuff after that
<jtv> danilos: let me just help colin out here, and then I'll get to your branch
<jtv> Or maybe the other OCRâ¦ oh.
<danilos> jtv, sure, thanks
<danilos> :)
<jtv> Oh wow, I've worked with meta-lp-deps in the distant past.
<danilos> jtv, yesterday?
<cjwatson> jtv: going to try to get my head round your mail now
<jtv> Your considerable head.
<jtv> Shouldn't be too hard.
<stub> Can someone else run bin/test.py -vv test_garbo , and tell me if it hangs for over a minute on test_HWSubmissionEmailLinker and test_ObsoleteBugAttachmentPruner ?
<danilos> stub, sure, need it on latest stable/devel/db-stable/db-devel?
<stub> danilos: stable or whatever. I think it is an old problem nobody noticed.
<danilos> stub, running now, you got me confused for a moment with bin/test.py instead of bin/test :)
<stub> DWIM
<jelmer> poolie: btw, can you land my branches on lp:lptools, or should I apply for members?
<stub> Actually, needs to be stable or devel - I think a cherry pick from db-devel -> devel killed it.
<jelmer> *membership
<stub> danilos: ^^
<poolie> there's already a bug for https://code.launchpad.net/~ubuntu-branches/ timing out right?
<jtv> cjwatson: your change has landed.
<poolie> jelmer: i'll look
<danilos> stub, right, I am trying with stable
<jelmer> poolie: thanks :)
<poolie> you have the power!
<danilos> stub, I am having some troubles with tests actually failing
<stub> danilos: database in use errors?
<danilos> stub, nope, LaunchpadSilentFailure or something :/
<danilos> let me check my dependencies
<danilos> (it's likely my download-cache is out of date)
<cjwatson> jtv: thanks!  and replied (though whether you'll like the reply I'm not sure ...)
<jtv> danilos: can't the preloading you're doing be done with load_related?
<jtv> cjwatson: argh!
<jtv> danilos: thinking specifically of do_eager_load (I understand you probably don't want to mess with that other code in detail)
<danilos> jtv, not that I know of, since I need to do a bunch of "get me this and then stuff that is related to it" (I did start with a bunch of load_related and load_referencing initially)
<danilos> jtv, i.e. I need double "related" indirection
<jtv> Can't you pass a result set to load_related?
<stub> So on stable, database/schema/trusted.sql no longer has _killall_backends - this removed in db-devel along with code changes. Looks like trusted.sql has ended up in stable without the code changes, which makes the db teardown rather flakey and slow for tests that used the librarian.
<jtv> danilos: ahhh, iswym
<jtv> you're starting out with a python-constructed nontrivial set of ids, not of objects.
<stub> oh... nm. it is still in stable. It is in testfuncs.sql, not trusted.sql
<stub> Running stable tests with a db-devel database... and it sometimes works!
<jtv> danilos: done
<danilos> jtv, thanks
<jtv> cjwatson: aren't ddtp-translations tarballs flushed from the librarian after a while?
<jtv> I mean, it's GC'ed automatically.
<jtv> And AFAIK fairly aggressively.
<jtv> (Also note that dsync isn't currently used.)
<cjwatson> jtv: surely not *that* frequently?  there's usually a ddtp-tarball upload not very long before release
<jtv> cjwatson: are we talking hours, days..?
<danilos> stub, ok, finally got it all to a clean state, I don't see the lag: https://pastebin.canonical.com/51339/
<stub> danilos: ta
<cjwatson> jtv: for natty it was about two weeks (https://lists.ubuntu.com/archives/natty-changes/2011-April/011309.html)
<cjwatson> jtv: but we could copy them if they're available and not worry if they aren't, I'd have thought
<cjwatson> jtv: dsync> that's a shame
<jtv> cjwatson: stub will correct me on this if needed but I think Librarian is on a much tighter GC schedule than that.  IIRC wgrant said flat-out that it would be impossible to do the copying for ddtp-translations.
<wgrant> jtv: ddtp-translations is probably not expired.
<cjwatson> jtv: what generates the md5sums index then?
<wgrant> jtv: Rosetta translations are.
<jtv> ahh
<cjwatson> lp_archive@cocoplum:~$ ls -l /srv/launchpad.net/ubuntu-archive/ubuntu/indices/md5sums.gz
<cjwatson> -rw-r--r-- 1 lp_publish lp_publish 29926782 Aug  8 10:23 /srv/launchpad.net/ubuntu-archive/ubuntu/indices/md5sums.gz
<jtv> Okay, so we could add ddtp-translations.
<cjwatson> that looks unfortunate
<jtv> However
<wgrant> cjwatson: It was generated by dsync.
<cjwatson> (I'm fairly sure that there exist mirroring tools which look at indices/md5sums.gz)
<stub> If the LibraryFileAlias is linked to another row in the DB, it is not garbage collected. If it is not linked, it will be removed in <72hours IIRC.
<wgrant> Which we should really running again.
<cjwatson> wgrant: yes, that's my point :)
<stub> We can change that timeout easily enough, although we pay for it in disk space.
<jtv> cjwatson: I have no doubt that it could be improved in lots of ways, but unless there's something decidedly wrong or unsafe I'd much prefer to get a partial solution in now and improve on it later, than drag this out further.  It's already sat around bit-rotting for longer than is usual nowadays.
<cjwatson> jtv: that's fair enough, but I would like to see those comments addressed at some point
<jtv> Absolutely.
<cjwatson> I wasn't intending these to be blockers, but you did ask for a review :-)
<jtv> But since this was originally billed as "if you think there's something you can do about it easily, you know, in a day or so..."
<cjwatson> I *think* your date comparison approach is safe, even if IMO not the right thing
<jtv> Well it was more a "if you know of any reason why this should not land, speak now or forever hold your peace" kind of thing.
<cjwatson> perhaps you can turn it into bugs
<jtv> Sure.
<jtv> I was thinking kanban cards, but I can do both.
<cjwatson> or that, I don't know your processes :)
<cjwatson> just avoid being forgotten
<jtv> An advantage of splitting these off is that other people can work on them, who knows, maybe even at the same time.  :)
<cjwatson> :-)
<cjwatson> thanks
<jtv> Glad to help.  If you'll let me.  :-P
<jtv> cjwatson: by the way, any idea where in LP these tarball names are already being parsed?
<cjwatson> jtv: lib/lp/archivepublisher/{debian_installer,dist_upgrader,ddtp_tarball}.py
<poolie> hello
<jtv> Great, thanks.
<poolie> if there's anyone on maintenance good at sql perhaps they could look at bug 827935
<_mup_> Bug #827935: consistent timeout on user branch listing page <Launchpad itself:New> < https://launchpad.net/bugs/827935 >
<cjwatson> which is what processes those uploads in the first place
<cjwatson> # This code is mostly owned by Colin Watson and is partly refactored by
<cjwatson> # Daniel Silverstone who should be the first point of contact for it.
<cjwatson> current comments 'r' us
<poolie> :)
<poolie> i love how many old names still appear in the lp test data
<poolie> daf, daniels,
<poolie> stevea
<cjwatson> (custom uploads used to be processed entirely by hand.  it was horrendous)
<jtv> cjwatson: and you're saying that the first parameter in the tarball name is not needed for disambiguation?  It can change in the next upload while still being a complete replacement for the current one with a different prefix to the tarball name?
<jtv> I've been assuming that architecture _if present in the tarball name_ does disambiguate, i.e. a ppc dist-upgrader does not replace an i386 dist-upgrader with otherwise the same tarball name.
<jtv> And that if there's no architecture, that means "all."
<cjwatson> the upload code disambiguates the tarball type by a field in the .changes file ('raw-debian-installer' etc.) rather than by the tarball prefix.  Ideally that would be preserved somewhere in the database too, and I thought it was
<cjwatson> ISTR an enum for it
<jtv> Hmmm
<jtv> bigjools, do you know about this?  ^^
<cjwatson> all the current parsers just ignore the first component
<cjwatson> lib/lp/soyuz/enums.py:PackageUploadCustomFormat
<cjwatson> existence of that suggests to me that the type is stored in the db somewhere
<cjwatson> though I guess that could just be queue rows
<bigjools> it is somewhere
<cjwatson> if it isn't stored, then in practice you can probably get away with looking at the first component of the tarball name, since AFAIK they've never changed yet
<cjwatson> jtv: TBH I have no idea what would happen if we ever tried to upload architecture-dependent dist-upgrader tarballs
<bigjools> jtv: see PackageUpload.contains_*
<cjwatson> jtv: but certainly a powerpc version of the debian-installer images tarball does not replace an i386 one with otherwise the same tarball name
<jtv> cjwatson: PackageUploadCustomFormat just distinguishes between debian-installer, ddtp-translations, dist-upgrader, rosetta-translations.  Much coarser.
<cjwatson> jtv: then I do not know what you mean by "the first parameter in the tarball name"
<bigjools> which uses PackageUploadCustom.customformat
<jtv> So again, not what we're looking for.
<jtv> The PackageUploadCustom is already fully accounted for.
<bigjools> can you re-state the question in simple terms then so I don't have to wade through backscroll
<cjwatson> the tarballs are debian-installer-images_VERSION_ARCH.tar.gz, dist-upgrader_VERSION_ARCH.tar.gz, or translations_COMPONENT_VERSION.tar.gz; I assumed you meant the "debian-installer-images", "dist-upgrader", or "translations" part when you said "the first parameter"
<jtv> OK: if we get two custom uploads of the same type for the same distroseries, what information should I take into account to determine whether the one supersedes the other, or is a separate thing that may also need to be copied?
<jtv> And I understand that this differs by custom type, but I'd like to know the specifics for debian-installer; dist-upgrader; and ddtp-translations.
<cjwatson> For debian-installer-images and dist-upgrader, different version + same arch means it supersedes.  For translations, different version + same component means it supersedes.
<cjwatson> i.e. we want the most current version of (debian-installer-images/per-arch, dist-upgrader/per-arch, translations/per-component)
<jtv> OK
<bigjools> jtv: in lp/archivepublisher/ there are custom file handlers for publishing
<jtv> That's not too far off what I have now, except (1) I currently don't have support for ddtp translations and (2) I currently also disambiguate by the "name" part (e.g. "debian-installer-images" vs. "dist-upgrader").
<jtv> bigjools: yes, colin just pointed those out.
<bigjools> jtv: they are called via PackageUploadCustom.publish...
<bigjools> the whole custom file thing is hideous
<jtv> Does that logic encode anything about what supersedes what?  If so, it may be better to extract and reuse.
<cjwatson> yes, to some extent
<bigjools> it does, somewhere, trying to find it
<cjwatson> e.g. I pointed to the fixCurrentSymlink stuff which does version comparison
<bigjools> right
<cjwatson> jtv: if the name changes, then the version will also change, so there isn't really much point in disambiguating on it
<cjwatson> this is guaranteed because if you don't change the version then you don't get a new directory under /dists/$SERIES/main/$BLAH/
<jtv> That's fine then.  It's just that for one of the upload types I ended up not supporting (translations I guess), the name part seemed to be a package name.
<cjwatson> no, it's NAME_COMPONENT_VERSION in that case
<cjwatson> where NAME == "translations" (not enforced/required but in practice true right now), COMPONENT in ("main", "restricted", "universe", "multiverse")
<cjwatson> that's the ddtp-tarball type
<cjwatson> rosetta-translations might have a package name there, but we don't care about rosetta-translations for this because they aren't published to dists/
<jtv> So only custom type & architecture matter for now, and for ddtp, component will also matter.
<jtv> Ah, I have some examples in the database now.  That helps.
<cjwatson> jtv: and ddtp isn't per-architecture, *only* per-component
<cjwatson> I expect at least part of the code for this ought to live in the DebianInstaller etc. classes ...
<jtv> Well I treat "no architecture" as "all" for genericity.
<cjwatson> err, DebianInstallerUpload I mean
<cjwatson> that sounds like an encapsulation bug, if you're having to consider architecture in generic code that suggests that actually the decision ought to be pushed down to the classes that know about the specific upload format
<cjwatson> IYSWIM
<jtv> Nah.
<nigelb> mrevell: Hi!
<jtv> The point is to isolate the format-specific part, so that it only extracts enough information to group the uploads such that of each group, only the latest matters.
<henninge> jtv: here is a fun branch to review for you  ;-) https://code.launchpad.net/~henninge/launchpad/devel-update-testtools-r228/+merge/71659
<jtv> henninge: sorry, past eod
<mrevell> nigelb, Hi, on the phone just now.
* henninge changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: danilos | Critical bugs: 238 - 0:[#######=]:256
<henninge> jtv: np
<jtv> Thanks.
<henninge> danilos:: here is a fun branch to review for you  ;-) https://code.launchpad.net/~henninge/launchpad/devel-update-testtools-r228/+merge/71659
<wgrant> jtv: Regretting your evil endeavour yet?
<nigelb> mrevell: cool, can you please poke me once youre done? :)
<jtv> wgrant: not at allâit turns out not to be impossible at all and my design can stay largely intact.
<nigelb> Seeing how much wgrant is failing to disappear from here during vacation, I'd suggest a +b from #launchpad-* during vacation time ;)
<jtv> Ooooh good idea
<wgrant> I only drop in occasionally!
<nigelb> where ocassionaly = ~2 hours
<jtv> â¦and then stay for how long, would you say?
<jtv> bigjools, cjwatson: different concernâ¦  I notice that the uniqueness check for debian-installer uploads distinguishes between uploads with ".0." in them and ones without!
<cjwatson> jtv: true, although we haven't used that for years
<cjwatson> perhaps we should just ditch that, it is a rather bizarre way to do it
<jtv> Aye, that it is
<jtv> Question is, what should be done about it?
<cjwatson> in fact I don't think we've used it since we switched to LP
<cjwatson> it dates from the similar dak code
<cjwatson> so if it's causing problems I suggest just binning it
 * jtv files separate bug
<nigelb> Does rocketfuel setup a password for postgres after I install postgres with that?
<cjwatson> I mean, in theory I suppose daily builds of d-i would be kind of nice ... but they were a lot more useful in the early days than they are now, and in practice just uploading d-i again whenever I need to works out fine
<cjwatson> definitely not worth awkward infrastructure to support it
<danilos> jam: hi, do you think you could have a go at https://code.launchpad.net/~danilo/loggerhead/bug-718566/+merge/71849
<danilos> henninge, looking at your branch now
<jtv> nigelb: it configures a trusted TCP login from localhost.
<henninge> danilos: thanks
<nigelb> jtv: so, can I safely do sudo passwd postgres and not break my lp install?
<jtv> nigelb: oh, a system password for the postgres user?  I don't see any reason why it should have a password at all.
<nigelb> oh.
<jtv> nigelb: sorry, I thought you were talking about a password for logging into the postgres databases.
<jtv> System users generally shouldn't have passwords.  No good for anyone but attackers.  :)
<nigelb> I'm just following instructions for hacking on harvest, which I think maybe a bit overzealous
<danilos> henninge, that was an easy one, r=me :)
<jam> danilos: commented
<henninge> danilos: indeed. Thanks ;-)
<danilos> jam: thanks, I'll try it out on a LP branch (I suppose you are suggesting doing it locally)
<jam> yeah
<jam> note that the *first* request is going to be uncached as well
<jam> since it has to do the work
<jam> my feeling is that it is already fast enough that we don't need the cache at all anymore
<danilos> jam: yeah, I can't really tell the difference between the two (trunk and the branch with removed changes), I'll do something more scientific from the terminal :)
<jam> danilos: probably try to find a revision with a particularly large number of changes
<jam> like when you merge into db-devel or whatever it is called now
<danilos> jam: right, I'll do that, and use "ab" to get actual data
<danilos> jam: fwiw, I got the following exception with trunk when benchmarking: https://pastebin.canonical.com/51341/
<jam> danilos: I've seen ScopeReplacer issues when running a lot of multithreaded requests as the "first" requests on a new loggerhead startup
<jam> danilos: so you can try starting loggerhead, making a single request
<jam> then running ab
<danilos> jam: right, thanks
<jam> ScopeReplacer happens at 'import' times, and it is known not to be threadsafe
<danilos> ah, makes sense, should I file a bug about it somewhere?
<LPCIBot> Project devel build #979: FAILURE in 5 hr 23 min: https://lpci.wedontsleep.org/job/devel/979/
<jam> danilos: there is a bug already
<danilos> jam: cool, I've added benchmarking results with a summary on https://code.launchpad.net/~danilo/loggerhead/bug-718566/+merge/71849
<danilos> jam: tell me if you think I should land this considering the "mild" performance degradation
<jam> danilos: going from the Launchpad discussion. Which is "things MUST complete in <9s (or 5s, ymmv), and SHOULD complete in <1s". We are still well underneath satisfying the 5s rule, and even the 1s rule.
<jam> btw, I wouldn't expect /changes to differ
<jam> let me grab a URL, just a sec
<danilos> jam: sure, I wasn't sure if /changes used this cache as well
<jam> it is the +revlog URLs that would be affected, I think
<jam> http://bazaar.launchpad.net/+branch/bzr/+revlog/pqm@pqm.ubuntu.com-20110817084016-600z65qzqmmt44w7
<jam>  /changes doesn't show you the file changes until you expand something
<danilos> jam: ok, I'll test it directly as well
<jam> danilos: updated the mp with my comment
<jam> I think I have a URL for you to try
<jam> (posted to the mp)
<jtv> cjwatson: see email.  All that needed immediate fixing then was that the "name" part of those tarball filenames is irrelevant, right?  I just fixed that.
<jtv> (And cross-referenced to all those bugs I filed, *pant*)
<danilos> jam: right, I was already benchmarking that URL
<jam> danilos: I'm expected that to regress approximately the same as /revision, which would still put it in the "fully acceptable" realm.
<danilos> jam, that's right, 325ms vs 267ms for the trunk
<danilos> (trunk with cache populated, already accessed the page from the browser first)
<jam> danilos: right. And ultimately a cache will always be faster, so what is "enough". And I think using SQUID to cache these pages is far superior to trying to do a file-change cache.
<jam> you might also try one more URL
<jam> something with a small change
<jam> which should be much less sensitive
<danilos> jam: yep, I'll do that too
<rvba> danilos: Hi, can I add this MP to your queue? https://code.launchpad.net/~rvb/launchpad/desc-header-827893/+merge/71842
<danilos> jam: anyway, smaller revision is actually faster in the few runs I did (259ms vs 264ms), so all good
<danilos> rvba, a queue, there's a queue? :)
<jam> danilos: good deal
<rvba> danilos: :)
<cjwatson> jtv: yep
<cjwatson> jtv: thanks for the attention to detail :-)
<danilos> rvba, nice, simple and clean, r=me (and thanks for the formatting fixes as well)
<rvba> danilos: Thanks!
<danilos> jam: now, how do I go land loggerhead changes? just merge into trunk (I see nobody is using [r=...] tags)
<jam> danilos: yeah, merge it to trunk, comimt
<jam> then make a proposal against Launchpad to use the new loggerhead code.
<jam> utilities/source_deps I believe
<StevenK> utilities/sourcedeps.conf
<StevenK> jam: ^
<deryck> Morning, all.
<StevenK> stub: If you're still around, can you please re-rubber stamp https://code.launchpad.net/~stevenk/launchpad/denorm-bspph/+merge/71811 ? I've had to resubmit due to the addition of a pre-req branch.
<deryck> abentley, let's chat about this shortly:  https://code.launchpad.net/~deryck/launchpad/bug-heat-wonky-recent-activity-check-688364/+merge/71756
<deryck> abentley, and thanks! :)
<abentley> deryck: np
<deryck> abentley, I'm free now, if you have a minute to chat about that MP.
<deryck> abentley, mumble 1 o 1?
<abentley> deryck: sure
* danilos changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 238 - 0:[#######=]:256
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: abentley | Critical bugs: 238 - 0:[#######=]:256
<LPCIBot> Project devel build #980: STILL FAILING in 2 hr 25 min: https://lpci.wedontsleep.org/job/devel/980/
 * barry sees lots of timeouts today
<rvba> deryck: Hi, do you have a few minutes for a small JS question?
<LPCIBot> Project devel build #981: STILL FAILING in 31 min: https://lpci.wedontsleep.org/job/devel/981/
<barry> has anybody else noticed a high number of timeouts today?
<nigelb> mrevell: are you free-er?
<benji> Is anyone else seeing "ImportError: No module named html5browser" on devel?
<benji> the last buildbot run that wasn't killed by a KeyboardInterrupt was clean and I don't see any bugs filed about it
<barry> here are two urls that are consistently timing out for me today, but were fine yesterday:
<barry> https://bugs.launchpad.net/ubuntu/+source/zeitgeist/+bug/807950
<_mup_> Bug #807950: zeitgeist-daemon crashed with LookupError in remove_from_connection(): <_zeitgeist.engine.remote.RemoteInterface at /org/gnome/zeitgeist/log/activity at 0xb74ee2cc> is not exported at a location matching (None,None) <apport-crash> <bugpattern-needed> <i386> <oneiric> <running-unity> <unity-2d> <zeitgeist (Ubuntu):Triaged> <zeitgeist (Ubuntu Oneiric):Triaged> < https://launchpad.net/bugs/807950 >
<barry> https://launchpad.net/ubuntu/+search?text=chromium-browser
<mrevell> Hi nigelb, sure. How can I help?
<nigelb> mrevell: I'm looking to see if you have suggestions on how the subscription could be improved
<nigelb> I found it non-intuitive
<deryck> rvba, sure.  voice or chat?
<nigelb> fixing it would be trivial, but the UX is probably of more concern here than the actual patch
<benji> the ImportError can be reproduced with "bin/test -c -t test_archivesubscribers_index"
<nigelb> *comparitively trivial
<rvba> deryck: great, I think chat is ok ... I'll paste my understanding of the problem ..
<timrc> calling addArchiveDependency() on PPA's (belonging to my LP user ~timrchavez) via the web service is resulting in OOPses in the web UI
<rvba> deryck: It's weird problem with FormOverlay. Basically, when the overlay is closed (for instance 'cancelled') it stays in the page (which is normal) but since it's only hidden using "visibility: hidden" it prevents me from clicking on the radio button that is underneath it.
<rvba> Here is the real example:
<rvba> If you open up a row on this page https://dogfood.launchpad.net/ubuntu/oneiric/+localpackagediffs then changed the "Ignored" status using the radio buttons, you'll get an overlay to enter a comment (to describe *why* you changed the ignored status). Cancel the overlay and you will see that now you can't click on the radio buttons any more.
<rvba> I'm not the only one to have this problem it seems: http://stackoverflow.com/questions/1751861/do-controls-in-firefox-receive-mouse-events-when-their-css-visible-property-is-fa
<rvba> This fixes it: http://paste.ubuntu.com/668072/
<rvba> deryck: But obviously I don't want to mess with that without the advice of a JS expert ;)
<mrevell> nigelb, Subscription to what? Sorry, I've not really been following this channel much today.
<deryck> rvba, reading back, thinking.....
<nigelb> mrevell: Oh, I should have been clearer. I meant bug subscription, more specifically, subscribing  myself to a bug
<timrc> Here is a script that should allow you to reproduce the problem (replace the PPA with a PPA you own): https://pastebin.canonical.com/51356/
<deryck> rvba, yes, display none is what I would prefer.  I always wondered why we used visibility anyway.
<deryck> rvba, I couldn't think of this exact issue, but I knew there was something with invisibility vs display like this.
<bigjools> timrc: an OOPS code is more useful
<timrc> bigjools, certainly, OOPS-2055AU68
<bigjools> thanks
<rvba> Looks much saner to me to use "display: none" too ... I'll do the change thenÂ ... I hope it won't break other overlays ;)
<bigjools> timrc: FTR you can't add 2 dependencies on Ubuntu like that
<bigjools> just the latter one will do
<timrc> bigjools, it was originally a test to verify that... however
<timrc> bigjools, the first call to add the dep is what blows these ppas up
<rvba> deryck: thanks for looking into it.
<bigjools> ah
<timrc> bigjools, just did it again on a ppa owned by ~oem-archive user, OOPS-2055AR56
<bigjools> I need to wait for the oops to sync
<deryck> rvba, np
<timrc> bigjools, our goal is to try to figure out how to get this option via the web service, http://i.imgur.com/ewHzM.png
<timrc> bigjools, maybe you know off hand :)?
<timrc> not specifying any component gives us the second option
<bigjools> timrc: it defaults to that, no?
<timrc> bigjools, when we call addArchiveDepedency without a component specified, it defaults to "Use all the same components..." I believe
<bigjools> timrc: don't call addArchiveDependency at all, it defaults to use everything
<timrc> bigjools, but need to chang the Ubuntu Dependency...
<timrc> bigjools, we need to set the equivalent of "Basic" from "Default" in the web ui
<bigjools> timrc: ok your image paste didn't make that clear
<timrc> bigjools, sorry :)
<bigjools> ummm
<bigjools> not sure you can change this over the API tbh
<timrc> http://i.imgur.com/aDHjJ.png
<timrc> bigjools, would you consider this a bug then?
<bigjools> timrc: let me investigate a little
<mrevell> nigelb, We have a bug report ... bug 816105 ... I agree it's not easy to find the "subscribe me" option right now
<_mup_> Bug #816105: UI for subscribing to a bug is confusing <bugs> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/816105 >
<mrevell> nigelb, Was there something else you found non-intuitive?
<nigelb> mrevell: the other bits are great. I'd love to fix the bug if there's a consensus on what needs to be done.
<mrevell> nigelb, I'm about to go into another phone call ... I'd love to read your thoughts in a comment on that bug and then talk to you again later int he week, if that's okay.
<nigelb> mrevell: that sounds good.
<mrevell> great :)
<timrc> Two bugs, 1) using addArchiveDependency() is causing OOPses (which seems like a regression); and 2) Cannot express "Use all components availalbe" via the web service for addArchiveDependency()
<timrc> bigjools, ^
<cody-somerville> bigjools, https://lp-oops.canonical.com/?oopsid=OOPS-2055AZ51
<adeuring> abentley: fancy a "double review"? https://code.launchpad.net/~adeuring/lazr.batchnavigator/lazr.batchnavigator-slicing-only-in-factory/+merge/71913 and https://code.launchpad.net/~adeuring/lazr.batchnavigator/lazr.batchnavigator-slicing-only-in-factory/+merge/71913
<abentley> adeuring: sure, I'll have a look.
<adeuring> abentley: one link should be https://code.launchpad.net/~adeuring/launchpad/bug-739052-6/+merge/71912
<bigjools> timrc: grar, ok please stop adding those, we need SQL to clean the DB
<bigjools> timrc: can you get me a complete list of PPAs where you've done this and I will wipe all their dependencies
<timrc> bigjools, lol, okay
<timrc> bigjools, https://pastebin.canonical.com/51366/
<allenap> matsubara: Are you the person to talk to about qa-tagger?
<matsubara> allenap, I'd be happy to talk to you about it. Anyone in the maintenance teams is encouraged to fix bugs there too.
<bigjools> timrc, cody-somerville: FYI: https://bugs.launchpad.net/launchpad/+bug/828133
<_mup_> Bug #828133: Archive:+edit-dependencies OOPS <oops> <ppa> <Launchpad itself:Triaged> < https://launchpad.net/bugs/828133 >
<allenap> matsubara: Ah, okay. Well, I'm wondering if it has missed a bug. The branches in https://bugs.launchpad.net/launchpad/+bug/824499 seem to have landed in stable but qa-tagger has not updated the bug. I'm trying to figure out why.
<_mup_> Bug #824499: "All derived distros" option for publish-ftpmaster <derivation> <Launchpad itself:In Progress by jtv> < https://launchpad.net/bugs/824499 >
<timrc> bigjools, this bug was Fixed Release, so it seems to imply the interface should be usable for adding Ubuntu deps... https://bugs.launchpad.net/launchpad/+bug/776449, no?
<_mup_> Bug #776449: Set Ubuntu dependencies for PPA via API <api> <bad-commit-13157> <escalated> <not-pie-critical> <oem-services> <ppa> <qa-ok> <Launchpad itself:Fix Released by abentley> < https://launchpad.net/bugs/776449 >
<allenap> matsubara: Ah, I think I've figured it out. The wrong bug number was used in the commit message.
<allenap> Sorry for the noise.
<bigjools> timrc: I forgot about that
<bigjools> timrc: I guess it has a bug
<timrc> bigjools, I should note that this was working (at least setting the Dep) up until recently (not sure when it went bad)... what wasn't working was the ability to achieve the first Components option in the web ui from the web service
<matsubara> np allenap
<abentley> adeuring: r=me
<adeuring> abentley: thanks!
<timrc> bigjools, here's the other bug
<timrc> https://bugs.launchpad.net/launchpad/+bug/828145
<_mup_> Bug #828145: Cannot add an archive dependency that uses all Ubuntu components available via the web service <oem-services> <Launchpad itself:New> < https://launchpad.net/bugs/828145 >
<bigjools> timrc: I think you misunderstand
 * bigjools writes on bug
<timrc> bigjools, probably :)
<timrc> bigjools, all I know is addArchiveDependency() will change the what's displayed in that view
<timrc> bigjools, so I expect to be able to do everything via the API that I can do in that view :)... is that too much to expect?
<bigjools> timrc: yeah :)
<bigjools> the addArchiveDependency does not allow you to change the ubuntu dependencies
<bigjools> just add new ones
<timrc> bigjools, when I create a PPA and use addArchiveDependency() to add an ubuntu dependency, it alters what's displayed in "Ubuntu dependencies"...
<bigjools> timrc: you're trying to change the ubuntu dependency to just "main" aren't you?
<bigjools> timrc: ah ok
<bigjools> right, I remember how it works internally now
<timrc> bigjools, here's our actual method... this will change the selection to the "Basic" Ubuntu dependency
<timrc> bigjools, http://paste.ubuntu.com/668400/
<timrc> bigjools, however, it assumes the wrong Components choice
<bigjools> timrc: see my comments on the bug
<timrc> bigjools, ah
<timrc> bigjools, works like a charm
<bigjools> great!
<timrc> cody-somerville, honestly a little surprised you didn't catch that... tsk
<timrc> :P
<mrevell> Hey, sinzui, have you seen bug 827902?
<_mup_> Bug #827902: Private teams not able to subscribe to bugs or blueprints for email updates <disclosure> <privacy> <teams> <Launchpad itself:Triaged> < https://launchpad.net/bugs/827902 >
<bigjools> abentley: you know you did that change so that PPA dependencies can be set on the API?  Can you have a look at this bug that it's causing:  https://bugs.launchpad.net/launchpad/+bug/828133
<_mup_> Bug #828133: Archive:+edit-dependencies OOPS <oem-services> <oops> <ppa> <Launchpad itself:Triaged> < https://launchpad.net/bugs/828133 >
<abentley> bigjools: I don't think I have any special insight.
<bigjools> abentley: it's extremely weird
<bigjools> the data looks sane
<abentley> bigjools: It looks like a lp.soyuz.model.component.Component is the input to the getTerm.  Is that correct?
<bigjools> abentley: should be, it's worked for ages with no changes
<bigjools> shame Component doesn't have a __repr__ :/
<bigjools> oh it does
<bigjools> so it doesn't get used when the security proxy is in the way
<cody-somerville> timrc, catch what?
<bigjools> abentley: I see what's wrong
<sinzui> mrevell, I have. I added the disclosure tag
<bigjools> abentley: the API allows you to add data that the form cannot cope with
<bigjools> timrc: set that component to "multiverse" and it'll be OK.
<timrc> cody-somerville, that by specifying 'multiverse' as the component, for example, it pulls in 'main', 'restricted', and 'universe' as well
<bigjools> the form code for the page only knows about multiverse and None :)
<abentley> deryck: Why aren't comments 8 and 9 showing? https://bugs.qastaging.launchpad.net/launchpad/+bug/1000
<_mup_> Bug #1000: There are too many bug reports in Malone <lp-foundations> <Launchpad itself:Invalid> <Ubuntu:Invalid> < https://launchpad.net/bugs/1000 >
 * deryck looks
<abentley> deryck: These are comments I created using the 'macintosh' charset, but I wouldn't expect that to hide them...
<deryck> abentley, hmmm, I have no idea.  I wouldn't expect them to be hidden either.
<deryck> abentley, I can get to them directly, too.  Weird.
<abentley> deryck: They are missing the Subject line, unlike 7 which was the same except it used the 'mac-roman' charset.
<deryck> abentley, ah, good catch.  curious that a missing subject line on an email generated comment would cause the comment to hide.
<abentley> deryck: The Subject was present on the email, though.
<deryck> abentley, right, I followed.  I assumed we tripped over it somehow and didn't store it in the db.
<abentley> deryck: Yeah, I suspect.
<deryck> abentley, seems like another bug that the comment won't display without a subject.
<abentley> deryck: I doubt it's just any missing subject.
<abentley> deryck: I bet it's related to the charset.
<abentley> deryck: I've sent a latin-1 email with absolutely no subject, to see what happens.
<deryck> abentley, good idea
<abentley> deryck: subject-free comment is not hidden.
<deryck> abentley, ok, good to know.  Thanks for confirming.
<abentley> deryck: I am marking bug 820039 qa-ok, and filing a new critical bug for "bug comments with macintosh encoding not displayed".
<_mup_> Bug #820039: process-mail.py fails with a LookupError: unknown encoding: macintosh error <mail> <oops> <qa-needstesting> <Launchpad itself:Fix Committed by abentley> < https://launchpad.net/bugs/820039 >
<deryck> abentley, sounds good.
<benji> abentley: do you have a minute for a review? https://code.launchpad.net/~benji/launchpad/bug-162868-2/+merge/71942
<abentley> benji: sure.
<benji> abentley: here's the soundtrack: http://www.youtube.com/watch?v=0O2aH4XLbto&feature=channel_video_title
<abentley> benji: The change looks reasonable.  The docstring on Macro is sort-of obsoleted by the branch.
<abentley> benji: it talks about the present state, rather than the state we'll have once the branch lands.
<benji> abentley: hmm, you might be right; I'll take a second look
<abentley> benji: e.g. "that pattern also has the side effect of making the template URL traversable from any object" could be "without this class, that pattern would make the template URL traversable..."
<abentley> benji: r=me.
<benji> abentley: sounds good, I'll change that;  I'd like to keep the docstring though, I like that it provides both a justification for why the class exists and describes how it should be used.  I often wish someone had written something like that when reading code.
<abentley> benji: yes, I agree.
<benji> cool
<abentley> benji: You might want to update this page: https://dev.launchpad.net/Web/TemplateCodeReuse
<benji> abentley: ooh, thanks
<sinzui> jcsackett, mumble?
<jcsackett> sinzui: sure, one moment.
<lifeless> allenap: hi
<lifeless> allenap: I've commented on the review; I have to run now to a hospital visit; I hope I haven't been too brief with my description.
<lifeless> allenap: if I have, please ask here or there and we can refine our understanding of whats going on
<lifeless> ok, popping out to my monthly allergy injection, at l'hopital, back in ~ 3hr
#launchpad-dev 2011-08-18
<lifeless> can has review ? https://code.launchpad.net/~lifeless/python-oops-wsgi/extraction/+merge/71969
<lifeless> mwhudson: btw - ^ django is wsgi based yes? you might like that :)
<wallyworld> lifeless: ^^ as a drive by, did you want to fix the ordering of the imports and __all__ declaration?
<lifeless> wallyworld: I can, I didn't notice they were unordered?
<lifeless> wallyworld: indeed, they look ordered to me
<wallyworld> lifeless: the __all__ is below the imports when it should be above?
<lifeless> thats correct
<lifeless> pep8 '      You should put a blank line between each group of imports.
<lifeless>       Put any relevant __all__ specification after the imports.'
<wallyworld> al of our lp py code is the other way around then
<wallyworld> ie it has __all__ first then imports
<lifeless> we should fix that :)
<wallyworld> that's a *lot* of fixes
<lifeless> yes
<lifeless> but easier than fixing all of pypi !
<wallyworld> hah
<wallyworld> i like lp's way of doing it
<wallyworld> easier to see what's exported
<jtv> Is the error reporting utility meant to be broken?  It seems to have killed dogfood.
<lifeless> jtv: broken? no. Changed? yes.
<lifeless> jtv: symptoms?
<jtv> Hang on, still pasting
<jtv> lifeless: http://paste.ubuntu.com/668808/
<jtv> Not too helpful, I'm afraid.
<lifeless> bwah
<lifeless> probably need to drop a pdb in there and have a peek
<jtv> Ouch.
<lifeless> I have just overhauled oops handling entirely
<lifeless> make sure your download-cache is up to date
<lifeless> and that you have bin/buildout
<jtv> Does it require a config change by any chance?
<jtv> I ran rocketfuel-get, which I think takes rare of the other stuff.
<lifeless> no, no config changes
<lifeless> jtv: can you try bin/buildout ?
<jtv> Hmm the tree ended up in an unbuilt state.  Rebuilding first.
<lifeless> wallyworld: did you have any other comments on that mp ?
<wallyworld> lifeless: no, looks ok to me. there's not much to it
* lifeless changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: -  | Critical bugs: 238 - 0:[#######=]:256
<jtv> lifeless: dogfood's running now.  Does rocketfuel-get not run buildout?
<lifeless> not directly
<lifeless> it runs make
<lifeless> rocketfuel-get is unmaintained AFAIK - you're the first person I've heard using it some time
<wgrant> It still works fine.
<mwhudson> i imagine it gets run very roughly once per someone joining the lp/buying a new machine?
<mwhudson> oh no, that would be -setup
 * mwhudson hides again
<jtv> Well, -setup runs -get.
<jtv> Now, why didn't "make" run buildout?
<lifeless> mwhudson: hey
<lifeless> mwhudson: so, did you see my oops stuff ?
<mwhudson> lifeless: i didn't click the link, i'm afraid
<lifeless> mwhudson: tsk!
<lifeless> mwhudson: what do you do for error reporting / aggregation / analysis with your dango sites?
<mwhudson> lifeless: currently embarrassingly little
<lifeless> mwhudson: so the goal of oops-* is to be easy to use.
<lifeless> mwhudson: *hint* *hint*
<mwhudson> lifeless: i believe there's a plan to deploy http://pypi.python.org/pypi/django-sentry
<lifeless> sentry is interesting
<lifeless> isd are evaluating it
<lifeless> it really operates on single log entries FWICT
<jtv> lifeless: are you reviewing today?  I've got a small one here: https://code.launchpad.net/~jtv/launchpad/bug-824553/+merge/71823
<lifeless> yes, and its done :)
<lifeless> jtv: I will happily review most any day
<jtv> Thanks.  Yes, I knowâbut I also know how much other stuff you do and I wonder if you have enough clones in that closet of yours...
<nigelb> lifeless: wha, rocketful-get is unmaintained? I keep using it!
<StevenK> nigelb: I do as well -- but the last thing rf-setup does is copy rf-get to /usr/local/bin. If there any other changes to rf-get later, you don't get them ...
<nigelb> ah.
<nigelb> isn't sort of easier to just modify $PATH?
<nigelb> that way the changes do get propogated.
<nigelb> or alternatively, symblink instead of copy.
<lifeless> grah
<StevenK> nigelb: And then when devel breaks, you can't use rf-get to fix it ...
<lifeless> ok, web.py is off the microservice list
<nigelb> StevenK: err, why?
<nigelb> or rather, why not
<StevenK> lifeless: What has it done? Or not done?
<lifeless> StevenK: it -very much- does not want oops_wsgi in the call tree
<lifeless> StevenK: it helpfully doesn't use wsgi internally!
<lifeless> StevenK: so there is nowhere in its stack to inject the middleware
<lifeless> StevenK: and if you warap the stack, it does its own error handling.
<lifeless> *headdesk*
<nigelb> try Flask? I don't know about Flask /that/ deep to know if it will work or not.
<nigelb> If there was some code without tests and I add some more code to it, do I write tests for the entire thing?
<nigelb> (I'm fairly sure the answer is yes, but I could try ;) )
<StevenK> You aren't required to, but it would be awesome.
<nigelb> Great! Can I bug you for help with it? ;)
<StevenK> If you like
<lifeless> nigelb: nope, flask is also overly clever.
<lifeless> nigelb: rather than having separate concerns, its monolithic,
<nigelb> yay ;)
<lifeless> I will port gpgverify to paste tomorrow
<nigelb> write your own wgi app then? :)
<nigelb> *wsgi
<lifeless> nigelb: no, we use paste for loggerhead and its great
<nigelb> :)
<lifeless> a little *too* bare bones in some ways, but it doesn't get in the way.
<wgrant> Also, it's not CherryPy.
<wgrant> That's the main thing.
<nigelb> heh
<wgrant> (Loggerhead used to use CherryPy. Those days were bad.)
<nigelb> So, what happens to IPerson(self.request.principal) when I'm not logged in?
<lifeless> None IIRC
<nigelb> hm, why is my code breaking then
<lifeless> ok, I'm kindof EODing, allergy injections suck :)
<nigelb> user=None for search should just work right?
<nigelb> halp.
<nigelb> TypeError: ('Could not adapt', <zope.principalregistry.principalregistry.UnauthenticatedPrincipal object at 0xb3f028c>, <InterfaceClass lp.registry.interfaces.person.IPerson>)
<nigelb> is it kosher to do a try catch?
<nigelb> wallyworld__: around? :)
<lifeless> should just be self.user if you're using a form, or call_with in the API
<wallyworld__> nigelb: hi
<lifeless> anything else and you're doing it wrong :)
<nigelb> lifeless: no no.
<wallyworld__> lifeless: it's not a form and it's not the ws api. it's a view
<nigelb> You warned me about the anonymous use case, only I don't remember how to catch it :)
<wallyworld__> similar to HugeVocabularyJSONView for example
<lifeless> blah, I meant view
<lifeless> LaunchpadView sets an attribute to Just Work
<lifeless> anyhow, I'm really not here, ciao.
<wallyworld__> nigelb: i think in the case of anon, request.prinicpal is None
<wallyworld__> nigelb: because LinkChecker doesn't extend LachpadView, there's no self.user
<wallyworld__> nigelb: perhaps you should make LinkChecker a subclass of LaunchpadView
<wallyworld__> it may then allow you to just use self.user
<nigelb> let me do that then
<wallyworld__> just a guess, it may work
<nigelb> wallyworld__: \o/ works :)
<wallyworld__> nigelb: excellent!
 * nigelb now needs to wwite tests
<nigelb> wallyworld__: how do I test behavior with a public bug number and a privaet bug number?
<nigelb> make the bug private and then try with anonymous user?
<nigelb> or something.
<wallyworld__> nigelb: you need to create a private bug and log in as the user who owns the project the bug is reported against
<wallyworld__> nigelb: with_person_logged_in is one way
<wallyworld__> nigelb: i assume you are talking about a python unit test here
<nigelb> so, create a person, create project with owner that person, create private bug, then logging as that user and try and try as anonymous
<nigelb> yah
<nigelb> wallyworld__: yeah, unit tests, I should have been further clear :)
<wallyworld__> yes what you say above sounds right. do it as 2 tests
<nigelb> always the fun bit in LP
<nigelb> unit testing
<adeuring> good mornning
<danilos> poolie, hi, if you are still around, are you familiar with our loggerhead QA/rollout setup?
<poolie> hi there
<poolie> danilos: slightly
<poolie> what do you want to know?
<danilos> poolie, well, I've landed a fix yesterday, and I would like how to best QA it, and how to ensure it gets rolled out
<danilos> would like *to know* :)
<poolie> oh, ok, i think i saw your mp
<poolie> do you know if that revision is live on qas?
<poolie> or staging?
<danilos> poolie, how can I tell that? :) (I assume that means it's not)
<poolie> one trick here is that there's only actually back-end storage for lp:bzr on qas and staging
<poolie> i think
<poolie> at any rate lots of branches are in the db but will break if you try to touch them
<poolie> for disk space reasons
<lifeless> pushing tp +junk is ideal
<danilos> poolie, right, I know that, but I should be able to push stuff to lp://staging/, right?
<poolie> danilos: so, does landing something into lp:loggerhead cause it to be rolled out?
<poolie> i don't think so but i'm not sure
<danilos> poolie, I have no idea, those are the things I am trying to find out, but I doubt it
<poolie> let's see if there's version info
<poolie> the losas are sprinting but they should be awake now
<poolie> spm?
<spm> poolie: hi
<lifeless> poolie: no, you need to update the version ref in lp
<poolie> spm, hi, can you tell me if normal deployments to qas pull in loggerhead from trunk?
<poolie> oh ok, i thought so
<lifeless> poolie: the qa deployment includes loggerhead, but not trunk
<poolie> danilos: so, i think you need to make an additional landing into lp:launchpad for it to be live, like that
<poolie> in fact it says 1.18 on the web page, and isn't trunk later than that?
<poolie> danilos: does that help at all?
<danilos> poolie, yeah, trunk does have a few revisions compared to what I have in sourcecode/loggerhead
<poolie> ok trunk still says it's v 1.18
<danilos> poolie, yeah, I figure I might need to land to loggerhead through PQM to ~launchpad-pqm/loggerhead/devel
<spm> poolie: sorry, quite distracted atm. have your questions been answered satisfactorily?
<poolie> yes, thanks
<poolie> enjoy your brown sauce
<spm> heh
<danilos> lifeless, do you know if I can just bump the LP loggerhead to trunk (I see issues like bug 826082 in a bit of indeterminate state), or if I should just include my own revision that I want rolled out?
<_mup_> Bug #826082: BadUrl from safe_open opening a branch <oops> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/826082 >
<poolie> danilos: i would like to get lp running trunk
<poolie> if necessary by reverting stuff that's unstable
<poolie> i don't have an detailed opinion about that specific bug
<danilos> poolie, so would I, but I ain't sure what are those
<danilos> poolie, so, I followed the trail of bug 820065 and bug 812583, which got me to the above bug
<_mup_> Bug #820065: TypeError: unsupported operand type(s) for -: 'int' and 'NoneType' <oops> <qa-untestable> <Launchpad itself:In Progress> <loggerhead:Fix Committed by pr0gg3d> < https://launchpad.net/bugs/820065 >
<poolie> well it looks like that might be fixed
<_mup_> Bug #812583: IndexError in add_template_values in loggerhead annotate <loggerhead> <oops> <qa-untestable> <Launchpad itself:In Progress> <loggerhead:Fix Committed by pr0gg3d> < https://launchpad.net/bugs/812583 >
<lifeless> so, we rolled back loggerhead after deploying coincided with a 1000 oops a day increase.
<lifeless> however
<lifeless> that increase appears to be jelmers safe-open refactoring
<lifeless> which he was fixing
<lifeless> In the spirit of one-change-at-a-time, I think we should get that sorted, then upgrade again.
<lifeless> if jelmers patch is landed, then upgraded loggerhead should be pretty straight forward
<danilos> lifeless, jelmer's patch is not landed in either loggerhead trunk or Launchpad
<poolie> he should be up soon and we can ask him
<danilos> (that would be clearer if it was "is landed in neither...nor" :)
<jelmer> it was in ec2 overnight but appears to have failed
<lifeless> danilos: it would be in launchpad
<danilos> poolie, lifeless, any suggestions on where should I put the docs for developers wanting to QA/deploy loggerhead? (something that'd be easy to find, I found no pages about it on dev.launchpad.net and wiki.canonical.com)
<lifeless> danilos: README.txt in loggerhead ?
<danilos> lifeless, I was more thinking in terms of Launchpad QA/deployment
<lifeless> danilos: I know its a bit LP specific, but you could note that this is the primary place its deployed :)
<lifeless> danilos: otherwise, yeah somewhere on d.l.n.
<wgrant> Ew, that does not belong in the loggerhead tree.
<danilos> lifeless, right, fair enough, though I'll add a few pages titled /Loggerhead to the dev wiki pointing at the readme then
<poolie> i think dev.l.n
<lifeless> danilos: its only a though, wiki is fine too
<lifeless> *thought*
<danilos> so, I guess d.l.n is the consensus choice here, thanks all for chipping in :)
<poolie> how to run or deploy it in general belongs in the tree; not the quirks of lp deployment though
<lifeless> the reason I think its reasonable to put it in the loggerhead tree is we want to run trunk
<poolie> if you ask me
<poolie> partly becaues there's much less chance losas will touch that
<lifeless> if we want to keep trunk deployable, we should have the instructions for that clear and visible to all contributors
<poolie> danilos: so, ping me or someone else if you want more
<danilos> poolie, lifeless: any fixes/improvements welcome on https://dev.launchpad.net/Code/Loggerhead, I am sure I got some bits of the process wrong
<danilos> I've also removed a reference to inactive project launchpad-loggerhead from dev.l.n/Code and pointed at this page instead
<danilos> jelmer, so, I assume I'd better wait for you to land your changes and get them deployed before I try that with my changes to loggerhead
<danilos> jelmer, I am sure you might have a few suggestions for https://dev.launchpad.net/Code/Loggerhead
<danilos> a buildbot failure in an unrelated test :(
<jelmer> danilos: Looking at the test failures right now..
<danilos> jelmer, cool, thanks
<danilos> jelmer, though, I was referring to a spurious buildbot failure for my branch :)
<danilos> lifeless, since I am lazy, what was the policy for triaging bugs about spurious test failures? critical? (bug 828584)
<_mup_> Bug #828584: Spurious test failure: test_sigint_exits_nicely <Launchpad itself:New> < https://launchpad.net/bugs/828584 >
<poolie> vila, jam, so i wonder how we could unstack it safely
<jam> poolie: reconfigure --unstacked isn't safe?
<jam> If you want the quick bzrlib invocation
<jam> just open the repository directly
<jam> repo.fetch(basis_repo)
<jam> which fills in everything
<vila> jam: without unstacking right ?
<jam> vila: if you do Repository.open() instead of Branch.open().repository
<poolie> jam, oh, i mean it _should_ be safe
<jam> the repo won't be stacked
<poolie> my question was really, is it safe, and how could we mitigate any risks
<jam> and .fetch() without a search recipe or target revision
<jam> fetches everything
<jam> poolie: I think it is safe, but if you want to do it easily Repo.fetch(repo) works just fine
<poolie> danilos: that looks pretty good; though i'd like to know whetehr things automatically go onto buildbot, *staging, etc
<poolie> i guess you would too :)
<jam> Speaking of which, I just did: jameinel@chinstrap:~$ time bzr branch lp:gcc-linaro/4.6 lp:~jameinel/gcc-linaro/test-4.6
<jam> Using default stacking branch /+branch-id/355386 at bzr+ssh://bazaar.launchpad.net/~jameinel/gcc-linaro/
<danilos> poolie, exactly :)
<jam> And it was still transferring at 500MB or so when I killed it.
<jam> but you know, that could be the "created a branch and didn't populate it" or something like that.
<jam> I'll try again with debug flags set
<jam> poolie: very strange "bzr branch lp:gcc-linaro/4.6 lp:~jameinel/gcc-linaro/test-4.6" is at about 1GB transferred so far.
<jam> 1.2GB
<jam> the discovery phase was fast, but the bytes transferred seems really wonky
<jam> there is only like 100MB in /4.6, so it doesn't line up at all, unless something is broken with the target stacking.
<jam> but the request says that it is trying to stack.
<jam> though this is bzr-2.1.4... because it was the binary on Chinstrap
<jam> not sure how much that matters or not.
<jam> It had "transferred" 1.2GB (probably down and then up again), and there was 465MB in the target. Still too much.
* allenap changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: allenap | Critical bugs: 238 - 0:[#######=]:256
<rvba> allenap: Can you please review this MP https://code.launchpad.net/~rvb/launchpad/overlay-bug-827220/+merge/71915Â ?
<allenap> rvba: Sure.
<rvba> allenap: : Thanks.
<lifeless> danilos: thats fine I think
<lifeless> I will look at the docs tomorrow :)
<allenap> rvba: Ah, interesting, I would have suspected that you'd need to use getComputedStyle(). r=me/
<danilos> lifeless, please extend them with as much as you know how stuff gets to *staging, what gets run in the buildbot and similar :)
<rvba> allenap: looks like getStyle is enough ...
<allenap> rvba: Cool.
<henninge> lifeless: Hi!
<henninge> lifeless: about python fixtures
<lifeless> hi
<henninge> lifeless: I don't see where we include it
<lifeless> henninge: setup.py
<lifeless> henninge: or do you mean inside LP? grep 'Fixture' lib -r.
<henninge> lifeless: yeah, in LP
<henninge> but I just saw that it is called "fixtures", let me look
<henninge> lifeless: ah ok, I was looking for python-fixtures
<lifeless> henninge: grep Fixture)
<henninge> it's in download-cache
<lifeless> will show where its used.
<henninge> no, I was not looking for that, I have used it in tests myself ...
<lifeless> at least directly; it won't show indirectly like in python-gbouncer, the rabbit notifier etc
<henninge> I just wanted to see which version we used
<lifeless> henninge: oh, :) I get you.
<henninge> lifeless: so you are saying fixtures will need an update to work with the latest testtools?
<henninge> s/update/patch/
<lifeless> yes, incompatible change to gather_details
<henninge> probably the same that I ran into
<henninge> because the output changed
<henninge> ?
<lifeless> no
<lifeless> rev 207 in testtools
<lifeless> breaks fixtures
<lifeless> fixed in rev 29 of fixtures
<henninge> lifeless: that's what I was trying to get at.
<henninge> so I don't need to fix fixtures, I just need to package the latest version of it.
<lifeless> wgrant: yes
<lifeless> bah
<lifeless> henninge: yes
<henninge> lifeless: thanks, that's much easier of course ;-)
<Ursinha> guys, anyone up to help me to understand what exactly launchpad understands when I say ubuntu.searchTasks(bug_supervisor=oneteam) ?
<jml> Ursinha: sure.
<Ursinha> afaik, packages don't have bug supervisors, only ubuntu does
<Ursinha> but using that returns me different tasks for different packages
<Ursinha> I can't ubuntu.searchTasks(structural_subscriber=oneteam) to compare, because that times out
<jml> Ursinha: just found http://paste.ubuntu.com/669193/ in bugtask.py
<Ursinha> bdmurray: ^
<Ursinha> jml: so it should work pretty much as structural_subscriber does?
<jml> Ursinha: I'm still grokking it. one sec.
<Ursinha> jml: sure
<jml> Ursinha: if I understand correctly, it'll match any package that oneteam is subscribed to
<jml> s/any package that/any bugtask on a package that/
<jml> as well as any bug task on ubuntu itself
<Ursinha> that's exactly what we need, and also what we tried to accomplish with structural_subscriber but it times out now...
<Ursinha> bdmurray: so we can use bug_supervisor
<jml> and it doesn't do team unrolling
<jml> exact match of user ids
<jml> https://bugs.launchpad.net/launchpad/+bug/191809
<_mup_> Bug #191809: A DistributionSourcePackage needs a bug supervisor role to control permissions <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/191809 >
<nigelb> mrevell: hi
<mrevell> Hey nigelb
<Ursinha> cool
<Ursinha> thanks jml :)
<nigelb> mrevell: hey, did you get a chance to look at that bug? :)
<jml> Ursinha: np
<mrevell> nigelb, Not yet. I'm at an event today. I'll be back tomorrow.
<nigelb> mrevell: cool, ok :)
<mrevell> gary_poster, nigelb is interested in bug 816105. As your squad worked on the feature, I'd love to hear your thoughts on that bug report.
<_mup_> Bug #816105: UI for subscribing to a bug is confusing <bugs> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/816105 >
<gary_poster> mrevell, any solution needs to support the new notification levels.  IMO, a solution should also communicate if "[y]ou have subscriptions that may cause you to receive notifications".
<gary_poster> A more perfect solution IMO (also fixing another related UI bug filed by mpt, I think) would also remove the necessity for the "Edit bug mail" link by making the notification of "[y]ou have subscriptions that may cause you to receive notifications" have a link to an in-page overlay that explains what is meant and what can be done about it other than muting.
<gary_poster> Specifically, this would be the "Other subscriptions" information on the "Edit bug mail" page included as an overlay on the bug page.  That information should only be loaded dynamically--it is somewhat expensive to calculate and +index is already slow enough.
<gary_poster> I suspect that the "Other subscriptions" code is factored well enough that this would not be too tricky to do.
 * nigelb reads the wall of text
<StevenK> nigelb: It's even more awesome when you talk to gary_poster IRL. Since then it's a wall of speech. :-P
<nigelb> StevenK: haha
<gary_poster> ha.  ha.  :-)
<nigelb> gary_poster: my main irk was this. it was way more easier to find how to subscribe others than subscribe myself.
<gary_poster> nigelb, I can see what you mean.
<wallyworld__> sinzui: both new picker feature flags are now on lp.net
<gary_poster> nigelb, I believe it is mrevell's responsibility to actually decide how it should work, or at least delegate and approve it.  The above is my take.
<nigelb> gary_poster: 'm still parsing that ;)
<sinzui> \o/
<wallyworld__> seems to work well
<gary_poster> the "wall" or the mention of mrevell as the responsible party as acting product mgr?
<mrevell> gary_poster, nigelb: Thanks both. I'll return to this tomorrow, when I'm back from the QA CoP sprint.
<gary_poster> cool
<nigelb> gary_poster: heh, the wall ;)
<gary_poster> nigelb, mrevell, the mistake we made IMO was not not take this part through user testing.  It was at the end of the project and we were in a rush.  Whatever is done, I suggest you not make our mistake, and actually produce mockups and run user tests.
<nigelb> gary_poster: I agree there. I'm only helping implement. Like I said the other day, I think the code bit is probably triivial. the UX bit is not.
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: allenap, jcsackett | Critical bugs: 238 - 0:[#######=]:256
<abentley> sinzui: I have a question about a distro in Launchpad: https://launchpad.net/cinux
<sinzui> ask away
<flacoste> hello everyone!
<deryck> Hi, flacoste.  Welcome back!
<flacoste> hi deryck!
<deryck> adeuring, hi, shall we mumble now?
<adeuring> deryck: ah, sure!
<bac> hi flacoste, welcome back to the land of the internets
<flacoste> hi bac!
<stub> Bah. We can't logout from the appserver layer as it redirects through secure codehosting to clear tokens before ending up on the openid side.
<stub> No tests for logout :-(
<gary_poster> welcome back flacoste
<flacoste> hi gary!
<henninge> deryck, abentley: I am really unsure with this javascript test. I am not sure how and what to test, maybe taht's why I keep finding excuses to do something else ... ;-)
<henninge> I have this now: http://paste.ubuntu.com/669304/
<abentley> henninge: looking.
<henninge> It's just checking that the handler works properly.
<abentley> henninge: Okay, do you want to mumble about it?
<henninge> abentley: sure
<cjwatson> Could somebody please try landing https://code.launchpad.net/~cjwatson/launchpad/dpkg-xz-support-619152/+merge/32868 ?  The necessary python-apt and apt-utils packages are now installed on production
<cjwatson> *try ... again
<sinzui> jcsackett, do you have time to mumble?
<abentley> allenap or jcsackett: could you please review https://code.launchpad.net/~abentley/launchpad/twisted-logging/+merge/72082 ?
<jcsackett> abentley: i'm otp now, but can look after that.
<abentley> jcsackett: thanks.
<jcsackett> abentley: looking at your branch now.
<abentley> jcsackett: great.
<abentley> sinzui: is bug 828493 related to your current work?
<_mup_> Bug #828493: Bogus results in package search <Launchpad itself:New> < https://launchpad.net/bugs/828493 >
<cr3> anyone know if there's a ui bug open against launchpad about the watermark and maincontent header being repetitive, like bugs.launchpad.net/checkbox showing "Checkbos System Testing" and "Bugs for Checkbox System Testing" respectively?
<sinzui> No, It is a duplicate of an old bug my team will fix
<sinzui> abentley, I will find the master and mark it as a dupe
<abentley> sinzui: rawk.
<jcsackett> abentley: looks good. r=me.
<abentley> jcsackett: thanks.
* allenap changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jcsackett | Critical bugs: 238 - 0:[#######=]:256
<sinzui> jcsackett, are you about to do a code review?
<mtaylor> spm: around?
<mtaylor> spm: if I did a staging import of bug a few weeks ago, and now I'm ready to go live, but re-ran the script to get more recent bugs - do we need to do another staging import? and do you want me to open a new ticket? or re-use the old one?
<sinzui> mtaylor, Another staging import is not needed, but someone, maybe spm, needs to run out preprocessor over your data to generate the bug nicknames. You need to post the URL to the updated data or send someone an email with the attachment.
<mtaylor> sinzui: k. awesome. https://answers.launchpad.net/launchpad/+question/168463
<mtaylor> sinzui: btw- I now have a script that can export bugs from a github project and generate the xml if you guys want a copy of it
<mtaylor> sinzui: https://github.com/openstack/openstack-ci/blob/master/gh_issues_to_lp_bugs.py
<mtaylor> sinzui: the lack of great way to map users and the lack of readily accessible email addresses sort of blows, but other than that it's workable at least
<sinzui> mtaylor, thanks
<sinzui> The user/email issue also happens when we import a mailing list. We can do it, but the feature was never exposed in the UI because of the troubles
<sinzui> I am creating the preprocessed xml file and updating the bug so that a system admin can do the real import
<mtaylor> sinzui: thanks!
* jcsackett changed the topic of #launchpad-dev to: https:dev.launchpad.net/ | On call reviewer: - | Critical bugs: 238 - 0:[#######=]:256
<lifeless> mtaylor: cool
<lifeless> jelmer_: hi
<lifeless> jelmer_: did the loggerhead fix land overhnight ?
<jelmer_> g'morning lifeless
<jelmer_> lifeless, r13726
<lifeless> excellent!
<lifeless> sinzui: around ?
<sinzui> I am
<lifeless> I'd like to understand this 'spying' thing more
<lifeless> whats the best way for me to do that ?
<sinzui> Since a private team cannot subscribe to a bug or branch you cannot
<sinzui> put if someone could. they could be subscribed to all of canonical's projects and the owners could not see that.
<lifeless> ok, so its not a feature. Its a potential antifeature ?
<sinzui> yess
<lifeless> I will follow up to the bug to note that
<sinzui> we talked about fixing the subscription rules for bug supervisor and security contact a few days ago. Doing that will address most of the issue.
<lifeless> how does that connect?
<sinzui> We would also need a rule when retargeting to a private project to change the subscribers.
<lifeless> ah
<lifeless> hmm
<lifeless> so the structural subs won't need any handling : they are per project anyhoew
<lifeless> explicit subs to the bug might need to be carried over
<sinzui> Bug.setPrivate() converts the structural subscriber to a real subscriber. the private subscription would not be visible
<lifeless> or ditched
<lifeless> setPrivate shouldn't do that
<lifeless> its a bug
<sinzui> yes, that was the gist of our conversation
<lifeless> ok, I'm assuming that that is fixed in this discussion :)
<lifeless> beyond that, we have explicit subscriptions to consider
<sinzui> I think the set-based rules to determine who should be subscribed on transitions to private/security will work.
<sinzui> We need to ensure those rules are in play when a bug it retargeted to a private project
<lifeless> yes
<lifeless> uhm, let me look for the other bug
<lifeless> it was something like 'need to ask who should be subscribed to a bug when it is made private'
<lifeless> which I think was duped on the strucsub issue, but actually shouldn't have been
<sinzui> The person who can make a bug private does not necessarily know who should be subscribed. I could subscribe someone who the owner do not want subscribed
<lifeless> the person who makes it private gets a subscription themselves.
<lifeless> they can then subscribe anyone.
<lifeless> so they can violate the owners desire anyway [given the current visibility rules
<lifeless> ]
<lifeless> ok, but it wasn't a dupe
<lifeless> it was assuming implicit subscriptions being carried across was broadly desirable.
<lifeless> so
<lifeless> I think there is a latent bug here but we can ignore it :)
<lifeless> sinzui: bug https://bugs.launchpad.net/launchpad/+bug/827902
<_mup_> Bug #827902: Private teams not able to subscribe to bugs or blueprints for email updates <disclosure> <privacy> <teams> <Launchpad itself:Triaged> < https://launchpad.net/bugs/827902 >
<lifeless> sinzui: my comments 2 and 3 - could you confirm I understand correctly?
<sinzui> lifeless, that is a good call on the rule to reveal the team's name
<lifeless> thanks !
<lifeless> its nice when a plan comes together
<lifeless> sinzui: and my neurons just rubbed together :) - cgregan linked the team in the report ;)
<lifeless> sinzui: so the rules we had previously put together are sufficient.
<lifeless> poolie: when do you want to catch up ?
<sinzui> fab
<lifeless> poolie: ping ;)
<poolie> hi lifeless, i was on the phone
<lifeless> no worries
<poolie> how about if i make some coffee and we talk in 15m?
<lifeless> sure
<cjwatson> Could somebody please try landing https://code.launchpad.net/~cjwatson/launchpad/dpkg-xz-support-619152/+merge/32868 ?  The necessary python-apt and apt-utils packages are now installed on production
<StevenK> cjwatson: Certainly.
<cjwatson> hurray
<cjwatson> (I have another one after this, but this one is probably simpler)
<sinzui> StevenK, wallyworld__ http://www.staff.uni-mainz.de/neuffer/scsi/fun.html
<lifeless> nice
<sinzui> wallyworld__, https://code.launchpad.net/~sinzui/launchpad/code-review-affiliation/+merge/72106
* wallyworld__ changed the topic of #launchpad-dev to: https:dev.launchpad.net/ | On call reviewer: wallyworld* (jtv) | Critical bugs: 238 - 0:[#######=]:256
<StevenK> cjwatson: Now that I'm off the phone -- your branch is in ec2, you should get a mail in ~ 4 hours.
<StevenK> cjwatson: Out of interest, has that branch been QA'd on dogfood before landing?
<cjwatson> it has not AFAIK
<poolie> hi lifeless
<cjwatson> StevenK: ^-
<cjwatson> StevenK: mawson doesn't yet seem to have the relevant apt-utils version, oddly
<cjwatson> StevenK: I'll whine at RT about that
<StevenK> cjwatson: mawson is a law unto itself, sadly.
<StevenK> cjwatson: I would be nice to QA it on qastaging, but asuka doesn't run any of the required services.
<cjwatson> (mawson == dogfood, right?)
<StevenK> s/I/It/
<StevenK> cjwatson: Right
<cjwatson> I've sent a whinge to RT#47132
<_mup_> Bug #47132: Documentation index not available -- yelp confused <bzr (Ubuntu):Fix Released> <bzr (Debian):Fix Released> < https://launchpad.net/bugs/47132 >
<cjwatson> TBH, for the .tar.xz branch, I can't see how it would break anything until/unless somebody uploads something .data.tar.xz-ish
<StevenK> cjwatson: Right, so my QA plan for the branch on mawson was upload a 'normal' package and then a different xz-using package and see that they both get processed correctly.
#launchpad-dev 2011-08-19
<cjwatson> actually, mawson has a sufficient python-apt version
<cjwatson> it'll have trouble with the multiarch-translations branch until we get it upgraded, but it'll be good enough for dpkg-xz-support
<cjwatson> so it can be QAed on dogfood
<StevenK> cjwatson: If this branch lands, it should be fine to test on mawson on Monday -- prod myself or wgrant before you start so we can update its code.
<cjwatson> in that case, can I get https://code.launchpad.net/~cjwatson/launchpad/multiarch-translations/+merge/67640 landed too?  we can test it on mawson then too
<cjwatson> (I don't know what the procedure for QAing db-devel branches is)
<StevenK> I think you might need to change the DB patch number. lifeless: ^
<lifeless> cjwatson: that needs to be split
<lifeless> its got both code and schema
<lifeless> and we only deploy schema from db-devel
<cjwatson> hm?  I already split out a schema branch for that
<lifeless> https://dev.launchpad.net/PolicyAndProcess/DatabaseSchemaChangesProcess
<lifeless> https://code.launchpad.net/~cjwatson/launchpad/multiarch-translations/+merge/67640 ? i see changes to lib/lp/archivepublisher/model/ftparchive.py, to === modified file 'lib/lp/archivepublisher/tests/apt-data/apt.conf'
<cjwatson> https://code.launchpad.net/~cjwatson/launchpad/multiarch-translations-schema which has landed
<lifeless>  etc
<cjwatson> multiarch-translations was supposed to be just the code bit
<lifeless> https://code.launchpad.net/~cjwatson/launchpad/multiarch-translations/+merge/67640 has a db patch in it - = added file 'database/schema/patch-2208-79-0.sql'
<lifeless> so this need to go to devel
<lifeless> and it can't land until we've deployed the schema change
<lifeless> thats blocked on the losas finishing pgbouncer migration
<cjwatson> I'm confused about why LP is showing that, if you look at https://code.launchpad.net/~cjwatson/launchpad/multiarch-translations-schema/+merge/67992 then it has the same schema patch
<cjwatson> and it's merged
<lifeless> which is blocked on them sprinting
<lifeless> its certainly not deployed
<cjwatson> I was told a couple of weeks ago that multiarch-translations needed to go to db-devel :-(
<lifeless> the schema did
<cjwatson> I would appreciate a consistent story here
<lifeless> the other part, *if* you land on db-devel, will sit there until you cherrypick it across to devel post deploy of the schema change.
<cjwatson> I can't cherrypick anything to devel personally, no access
<StevenK> 79-0 is already on db-devel
<lifeless> well, you would do it by preparing another branch which we can land via ec2 at the right time
<lifeless> cjwatson: if you want to flush your queue, you can land this on db-devel yes. But we can't deploy it from there,
<lifeless> cjwatson: I'm sorry that this feels inconsistent to you
<lifeless> cjwatson: particularly if its my fault :)
<cjwatson> 2011-07-14.log:12:54 <lifeless> cjwatson: for your branch, you'll need to land both components on db-devel (because we're not live on the incremental deploy steps yet)
<lifeless> cjwatson: ah yes, so thats up to me to be more clear
<cjwatson> so right now I am completely confused as to what I need to do.  I don't care about flushing my queue, I care about getting things deployed
<lifeless> ok
<StevenK> I can see the schema is already on db-devel
<lifeless> so right now, this is blocked on losa work: we have no way to deploy schema changes until the pgbouncer migration is completed.
<lifeless> thats stubs and the losas top priority ticket
<lifeless> as soon as thats done we'll start doing fastdowntimes, one per day, with one db patch per day
<cjwatson> that's fine.  what I'd like to know is (a) how I know when to proceed (b) what I should do next
<cjwatson> oh and (c) what if anythng I should do to the multiarch-translations branch
<lifeless> so - nothing.
<cjwatson> merge it from db-devel maybe?
<lifeless> the bug for the schema change should be sitting in fix committed now
<lifeless> when we have the patch deployed that will toggle to fix released
<lifeless> and we can continue with the rest of the work
<cjwatson> bug for the schema change> is that https://bugs.launchpad.net/launchpad/+bug/809123 ?
<_mup_> Bug #809123: we cannot deploy DB schema changes live <fastdowntime> <qa-ok> <Launchpad itself:Fix Committed by stub> < https://launchpad.net/bugs/809123 >
<lifeless> that too; no I meant the one linked to your schema patch
<lifeless> (there is one, isn't there? it may be the main one for your translations support)
<cjwatson> there isn't onoe
<lifeless> ah
<lifeless> ok
<beuno> hm
<lifeless> so i suggest picking this up when 809123 is fix released
<beuno> I'm getting an oops when searching for someone (trying to add a new ubuntu member)
<lifeless> at that point we'll be a few days out from your thing being live
<beuno> OOPS-2057DX
<cjwatson> ok, subscribing to that
<cjwatson> I basically just want to make sure that any delays in the process are not down to me - it's felt very much like there've been serial multiple-day delays so far
<cjwatson> and I appreciate the assistance in deployment but the feature in general is important enough that I need to report on t
<lifeless> cjwatson: welcome to LP dev :)
<cjwatson> *it
<cjwatson> heh
<lifeless> cjwatson: this is one of the things that we're working on reducing - dev friction is insanely high atm
<cjwatson> right, understood
<cjwatson> and FWIW non-db stuff has clearly been getting better from my POV
<lifeless> thanks!
<cjwatson> I've merged db-devel into multiarch-translations so that it now clearly only contains code.  Do I need to resubmit that against devel (presumably after 809123 is fix released)?  Can I do that without losing approvals or will it need re-rubber-stamped?
<lifeless> yes, an dyes, and probably needs a rubber stamp at that point. Use the 'resubmit' button.
<lifeless> (at the right time)
<cjwatson> ok
<jtv> Hi wallyworld__
<wallyworld__> jtv: yellow
<jtv> ?
<wallyworld__> jtv: hello
<jtv> Ah.
<wallyworld__> one review just popped up recently
<wallyworld__> doing it soon
<jtv> OK
<wallyworld__> just finishing off a ^@%%@!& yui test
 * jtv shudders in sympathy
<jtv> StevenK, are you hammering dogfood?
<StevenK> Not any more
<jtv> Ah.  I just updated the code/schema as well.
<jtv> And now it's failed to come up.
<jtv> Ah.  Not that again.
<jtv> Fixing it.
<nigelb> wallyworld__: I thought my feelings to yui was isolated. apparently not ;-)
<wallyworld__> nigelb: it's not that bad actually. just a bit tedious sometimes
<nigelb> I know!
<wallyworld__> to mock everything needed to allow the tests to run in isolation
<nigelb> that's one of the things I'm stuck at because I lack time.
<wallyworld__> how is your branch?
<wallyworld__> yeah, i can understand that
<nigelb> needs tests
<wallyworld__> well you are not alone there
<nigelb> hehe
<nigelb> and work's been busy, so I get very little time
<wallyworld__> what do yiu do for work?
<nigelb> er, what? :)
<wallyworld__> what's your day job?
<nigelb> ah, I work as a systems engineer at a local startup :)
<wallyworld__> sounds cool. i worked for a startup once, about 11 years ago
<wallyworld__> but they went bust
<wallyworld__> is everything working apart from the tests?
<nigelb> heh
<nigelb> yeah, the code's working for anon as well as non-anon user
<nigelb> and I tested with private bug too
<nigelb> so I want to catch alll that i my tests
<nigelb> I put some ground work in there
<wallyworld__> sounds good.
<nigelb> the javascript bits doesn't have a test right now
<lifeless> booyah
<lifeless> I have oopses from gpgverify.
<nigelb> So, I'll probably write a test for the entire code.  Also, because I want to learn how to do it
<wallyworld__> nigelb: well, yes it will be educational to do it for sure
<StevenK> s/educational/brain-damaging/
<nigelb> lifeless: congrats!
<nigelb> wallyworld__: yeah, that was my thinking :)
<nigelb> StevenK: bwahaha
<wgrant> lifeless: Paste?
<lifeless> web.py
<lifeless> doing $anything on thursday is daft.
<lifeless> rule 1.
<wgrant> I thought you'd discounted it.
<wgrant> Hah.
<wgrant> Right.
<lifeless> http://bazaar.launchpad.net/~lifeless/+junk/gpgverify/revision/11?start_revid=11
<wgrant> text/html... ew.
<lifeless> meh
<lifeless> that was cargo cloned; I should change.
<wgrant> It must not be used frivolously.
<wgrant> Like Twisted Web.
<wgrant> (it defaults to text/html... what could possibly go wrong)
<lifeless> or $all $web $frameworks
<lifeless> so does flask
<lifeless> and paste IIRC
<wgrant> Kill it.
<lifeless> for their errors
<mwhudson> so does django
<wgrant> Kill them all.
<mwhudson> +1 wgrant
<wgrant> Yes, let's just default to the most dangerous content type in the world.
<wgrant> Nothing bad could come of that.
<lifeless> fixed in 12
<wgrant> Thanks.
<wgrant> Although 'OOPS-ID' is not a standard HTTP header, AFAIK :P
<nigelb> picky :P
<wgrant> lifeless: We can't have some generic WSGI container launcher thing that does OOPS stuff, rather than having all this stuff in the core code of every service?
<wgrant> Like our scripts, it seems silly to have all these boilerplate wrapper scripts rather than a parameterised launcher.
<wgrant> I guess twistd is already a similar sort of thing.
<wgrant> But I think we should do it for more than just daemons :)
<lifeless> wgrant: well, if we have two * web.py services, we can have python-oops-webpy
<lifeless> wgrant: it should be!
<lifeless> wgrant: the problem is, as I said yesterday, web.py isn't actually wsgi
<lifeless> its kindof wsgo
<lifeless> same goes for flask.weurkzeug
<lifeless> s/\.////
<wgrant> :(
<lifeless> they both want to stop exceptions bubbling out
<lifeless> so they error handle themselves rather than as layers
<wgrant> Yay
<lifeless> by the time you're sniffing start_response for 500's and trying to reconstruct a long-gong sys.exc_info you've lost.
<lifeless> I think hooking in in the preferred for for any given framework is best.
<lifeless> thats the point of having a simple, flexible core, after all.
<lifeless> its just a shame that wsgi is so micro that few things use it.
<wgrant> It would be nice if there was a standard.
<lifeless> e.g. werkzeug uses request and response objects, which wsgi has no knowledge off (and response puns as a wsgi app... which is actually quite good)
<lifeless> but its request objects are thread local meta things
<lifeless> anyhow
<lifeless> I'm going to setup a project for gpgverify
<lifeless> shove the current code in there
<lifeless> teach it to handle detached sigs and use a regular form object at the same time
<lifeless> and then next week, deploy the biatch.
<lifeless> jamesh is playing with python-oops-wsgi, which django manages to have a sane way to integrate in
<wgrant> Excellent.
<lifeless> which is - huge - brownie points for django IMO
<wgrant> (are you keeping a reasonably small backlog of in-progress work now, given your impending incapacitation?)
<lifeless> we have two very modest features identified there to make it have parity with the used features of wsgi-oops
<lifeless> wgrant: as much as I can yeah
<wgrant> Great.
<lifeless> I wanted to unblock SOA before my life changes forever
<wgrant> Yes.
<lifeless> the one key thing for oops remaining is to teach oops-tools to use a repo interface rather than parsing itself
<lifeless> then we could switch it over to a different backend
<wgrant> That interface gives back a dict?
<lifeless> yes
<wgrant> The current repo implementation is just a directory containing date dirs?
<wgrant> So oops-tools would need lots?
<lifeless> we'd want to shuffle a bunch of stuff like getOopReport(id) from lp into python-oops-datedir-repo
<lifeless> we could model oops-tools needs as either a collection of repos
<lifeless> or as a repo with knowledge of many root dirs.
<wgrant> Right.
<lifeless> the main thing is to lift the conceptual stuff out of rfc822 parsing and up to useful types
<lifeless> with that done, we can switch to bson and iterate on things like messages and extra keys easily enough
<lifeless> which is a totally separate project.
<lifeless> relatedly, moving to cassandra might be unblocked if we take a map-reduce search approach rather than a pre-indexed one (for finding oopses by non-id)
<wgrant> Why can't we pre-index?
<lifeless> we can, with e.g. solandra
<lifeless> but if we want anything more sophisticated map-reduce is probably a good idea anyway
<wgrant> Yeah.
<wgrant> Killall solr
<lifeless> so starting with map-reduce would deliver the necessary conditions
<lifeless> whereas starting with solandra doesn't really.
<lifeless> Ã¾
<lifeless> anyhow
<lifeless> thats for tomorrow^Wmonday
<poolie> lifeless, hi, could we have a short supplementary chat?
<nigelb> g31
<nigelb> argh
<poolie> :)
* wallyworld__ changed the topic of #launchpad-dev to: https:dev.launchpad.net/ | On call reviewer: - | Critical bugs: 238 - 0:[#######=]:256
<henninge> Anybody merging stable into db-devel currently?
<henninge> No, that's not trivial for me.
<henninge> ^ Looks like something jelmer_ and bigjools could do.
<bigjools> seriously?
<henninge> rvba: I will request a deploy. If your QA is quick it can still be included. ;-)
<henninge> bigjools: I looked at bzr annotate and found your name and jelmer's for safe_open
<rvba> henninge: if you're talking about 820452 then I'm afraid the QA won't be quick. :(
<henninge> rvba: ok, np
<henninge> one more reason to request the deploy asap ;)
<rvba> Right :)
<jelmer_> henninge, I'll have a look
<henninge> bigjools, jelmer_ : I have a branch ready, so if you want to tell me how to fix it, I can do the mechanics.
<bigjools> henninge: have you fixed the conflicts?
<henninge> no ;)
<henninge> just done the merge and looked at them
<henninge> it's just one in each file, so it might be easy
<bigjools> ok
<StevenK> henninge: I'm a little concerned about deploying now, TBH.
<henninge> StevenK: why is that?
<henninge> StevenK: because it's Friday?
<StevenK> henninge: Because most of IS are going to be on a plane over the weekend
<henninge> oic
<wgrant> It's Friday and everyone will be in the air for the weekend.
<wgrant> And we don't have APAC LOSAs on Monday.
<wgrant> The first LOSA will appear in 72 hours.
<bigjools> do not deploy today, please
<StevenK> The earliest we can deal with problems is Monday UK
<henninge> good reasons ;-)
<bigjools> we never deply on Fridays
<henninge> I thought mornings are ok
<henninge> but the "in the air" issue is a neck breaker.
<henninge> I will re-visit this on Monday
<bigjools> mornings are not ok
<bigjools> some problems will only occurr overnight
<henninge> so I learn ;-)
<StevenK> henninge: Friday morning *in Australia* can be okay, since then we have LOSA coverage for another long while
<wgrant> (we will occasionally deploy up to early afternoon my time, but I normally try to stick to mornings)
<bigjools> henninge: for the test_notification conflict, take the second, larger chunk
<henninge> bigjools: I will
<bigjools> I dunno about the other
<henninge> jelmer_: any insight on the merge?
<henninge> for safe_open.py
<allenap> henninge: I just emailed the list that I would sort out the stable -> db-devel conflicts, but bigjools says you're on it, and so I shall leave you to it :) Thank you.
<henninge> allenap: you're welcome
<jelmer_> henninge: what's the conflict like?
<henninge> jelmer_: let me paste it
<henninge> jelmer_: http://paste.ubuntu.com/669867/
<jelmer_> henninge: you'd want the second chunk in that case
<jelmer_> (bzr resolve --take-other)
<jelmer_> is there a wiki page that describes what branch gets deployed where, when, how often, etc?
<henninge> jelmer_: thanks
<lifeless> poolie: sure
<cjwatson> argh, why does launchpad-dependencies hate me
<StevenK> cjwatson: It isn't just you, sadly.
<cjwatson> turns out that the upload target isn't a reliable way to test which version it's being built for either
<StevenK> cjwatson: I think I've just figured why your branch failed in ec2, too.
<cjwatson> so it's broken in lucid right now
<cjwatson> I suspect it might be related?
<cjwatson> I mean, I noticed while trying to upgrade my LP dev VM, but ...
<StevenK> cjwatson: The ec2 images are not updated for newer dependencies, they need to be done manually.
<cjwatson> I could have sworn I saw an apt-get dist-upgrade in the test setup
<cjwatson> ./lib/devscripts/ec2test/instance.py:134:aptitude -y full-upgrade
<cjwatson>  launchpad-dependencies (0.97~lucid1) natty; urgency=low
<StevenK> Hmm
<cjwatson> anyway, that's why it's going wrong
<cjwatson> bloody recipes
<StevenK> Hah
<cjwatson> while the IS builds are more like "launchpad-dependencies (0.97~0.IS.10.04) lucid-cat; urgency=low"
<cjwatson> Could somebody review/merge https://code.launchpad.net/~cjwatson/meta-lp-deps/fix-target-detection-harder/+merge/72166 ?  Reasonably urgent since launchpad-dependencies is currently broken in the LP PPA for lucid
<cjwatson> StevenK: maybe you could?
<StevenK> cjwatson: You mean distroseries, not distribution
<StevenK> But perhaps we should fix that
<StevenK> Either way, r=me
<cjwatson> StevenK: well - it's called Distribution in dpkg-parsechangelog output
<cjwatson> so I was mirroring that
<cjwatson> I can rename if you feel it necessary
<cjwatson> StevenK: can you land it for me?
<jtv> gary_poster: hi!  Say, do you know how I can enumerate all of Launchpad's registered utilities?  getGlobalSiteManager().registeredUtilities() gives me nothing.
<gary_poster> hi jtv.  Thanks for looking at that.  Lemme look, one sec.
<cjwatson> ... maybe somebody else can land https://code.launchpad.net/~cjwatson/meta-lp-deps/fix-target-detection-harder/+merge/72166 for me?  I've made the distribution -> distroseries substitution as suggested by StevenK
<gary_poster> jtv, I'm surprised that registeredUtilities is giving you nothing.  It looks like it does all the right things.  Could it be that you called it before zcml was processed?
<jtv> gary_poster: unlikelyâI'm calling it from a LaunchpadScript and one of the reasons was that zcml would be processed for me.  :)
<gary_poster> jtv, could you verify by trying to look up a utility?
<jtv> You mean using zope.component.getUtility?
<jtv> Or using getGlobalSiteManager().getUtility or something?
<gary_poster> jtv, both, either.
<jtv> gary_poster: Hmm... componentlookuperror
<gary_poster> jtv, that's what I would expect.  registeredUtilities really does look in some integral data structures. :-)
<jtv> Arrhâ¦ I think I needed to script.run() instead of script.main()
<jtv> Now it takes the requisite 20 minutes to start up.
<gary_poster> :-/
<gary_poster> cjwatson, that needs ec2 land I assume?
<jtv> gary_poster: it fails later now, so that gets us past the first hurdle.  Thanks.
<gary_poster> cool jtv, np
<cjwatson> gary_poster: no, AFAICS it's just direct-commit
<gary_poster> cjwatson, you mean, you can verify that all tests pass, and if buildbot fails you will be shocked, awed, and at least mildly embarrassed? :-)
<cjwatson> lp:meta-lp-deps is owned by ~launchpad-committers and the commits are all just direct
<gary_poster> of duh
<cjwatson> meta-lp-deps has no tests ...
<gary_poster> s/of/oh/
 * gary_poster was not looking closely enough
 * gary_poster goes to look at mp
<cjwatson> I wasn't suggesting direct commit to launchpad :-)
 * cjwatson is nowhere near that brave
<gary_poster> heh :-)
<gary_poster> cjwatson, ok, I'm giving it a whirl
<cjwatson> great, thanks
<gary_poster> cjwatson, please approve/adjust commit message: "[r=StevenK] fix target detection so that it works for both recipe builds and the CAT repository"
<jtv> gary_poster: next of my hurdles.  The docs I saw "on the internet somewhere" said UtilityRegistration had members "provides" and "provided."  But no.
<jtv> Maybe it's just a typo.
<gary_poster> jtv, according to code, it has "self.registry, self.provided, self.name, self.component, self.info, self.factory"
<cjwatson> gary_poster: ack, that's fine
<gary_poster> cool cjwatson
<jtv> gary_poster: yup, the provides was a typo.  I'm getting further now.
<jtv> Funny.  Some of them have empty names.
<gary_poster> cjwatson, done
<gary_poster> jtv, sure, those are utilities you just look up with an interface
<gary_poster> ILaunchBag, for instance
<cjwatson> gary_poster: thanks!
<jtv> Well lots of them give an interface name as the utility registration's name.
<gary_poster> welcome
<gary_poster> jtv, yes, that's something you will probably want to filter out.  IIRC, something or other registers all interfaces that it encounters as a utility.  So that it can do something else.  Later.
<gary_poster> s/all interfaces/each interface/
<jtv> gary_poster: it looks as if the ones _with_ names are the ones I'm supposed to skip.
<gary_poster> jtv, that's what I would expect to be the common case, yes.  I'd instead say that if the utility provides IInterface, skip it.  That might be equivalent, but it is more careful.
<jtv> Oh, OK
<jtv> gary_poster: checked 347 interfaces, 238 failed.  :/
<gary_poster> jtv, ow.  :-(
<jtv> Probably a bug in my code.
 * gary_poster hopes so, practically speaking :-)
<gary_poster> jtv, are you using the zope.interface-provided interface checker, or rolling your own
<gary_poster> ?
<jtv> gary_poster: I was lazy and grabbed our own test matchers' Provides.
<gary_poster> jtv, ah ok.  I think I looked at that once but I forgot how that works.  I meant zope.interface.verify.verifyObject FWIW
<jtv> Well, I'll just give it a whirl.
<gary_poster> cjwatson, we've got a problem.  Don't know if it is related... https://launchpadlibrarian.net/77594282/buildlog.txt.gz
<cjwatson> I just asked #is about that
<cjwatson> it's not related to my change
<gary_poster> ok thanks cjwatson
<cjwatson> I suspect badness due to puppet deployment but that's just a guess ...
<gary_poster> heh, puppet is getting a lot of bad press lately
<cjwatson> I may be blaming it unfairly, it's just that I think I heard of a previous instance of this problem and I only started hearing about it this wewek
<cjwatson> *week
<jtv> gary_poster: getting better â 109 failures (out of 347)
<jtv> I just wonder why so many raise DoesNotImplement.
<gary_poster> jtv, I suggest you change the checker to lambda *args: True
<jtv> What checker?
<gary_poster> jtv, sorry, trying to make a joke.  verify your interface with an "everything passes".  Hee.  Hee.
<jtv> Ah.  You areâ¦ American, right?  Might I suggest you check with a European first?  It may prevent misunderstandings.  :-P
<gary_poster> jtv :-P
<jtv> But the lambda idea is good too.  :)
<jtv> gary_poster: I wonder if I need to do this differently for classProvides than for implements.
<gary_poster> jtv, ah, quite possibly.
<gary_poster> jtv, so you mean that we are registering some classes as utilities, yes?
<jtv> And I'm pretty sure the component isn't supposed to be "security proxied __builtin__.type instance"
<jtv> Yes, we use both in LP.
<gary_poster> jtv, as a separate issue, you may need to rip off security procxies, if you have not already.
<jtv> With disturbing pleasure.
<gary_poster> lol
* adeuring changed the topic of #launchpad-dev to: On call reviewer: adeuring | Critical bugs: 238 - 0:[#######=]:256
<gary_poster> jtv, FWIW, I suspect you are right that neither verifyObject nor verifyClass will do what you want for class utilities.  You may need to make your own verification for that.
* adeuring changed the topic of #launchpad-dev to: https:dev.launchpad.net/ | On call reviewer: adeuring | Critical bugs: 238 - 0:[#######=]:256
<jtv> Seriously?
* bac changed the topic of #launchpad-dev to: https:dev.launchpad.net/ | On call reviewer: adeuring,bac | Critical bugs: 238 - 0:[#######=]:256
<jtv> But I've verified them in the past as verifyObject(II, getUtility(II))
<gary_poster> jtv, ah, then...I thought you implied it was not working? ("I wonder if I need to do this differently for classProvides than for implements.")  If it is working, then, uh, I suggest you keep using it.  I was just not confident that verifyObject would do what you need without digging into it.
<jml> why would you remove security proxies?
<jtv> gary_poster: I'm not sure if it's working or not.  I'm getting a bunch of DoesNotImplement exceptions (besides some other exceptions that I did expect)
<gary_poster> jml, primarily because security proxies will only let someone with the proper authority look at the value.  I suppose another approach would be to log in as a Launchpad admin (if I remember correctly that they have all permissions in all contexts).  Secondarily because the verification machinery might not expect them generally, and it's a good first pass to ignore them.
<gary_poster> jtv, ok.  In your shoes, I'd simply investigate some seemingly representative reports and see if the complaints seem to have merit.
<jml> gary_poster: oh, of course. I was thinking only of the security proxies other duty as JIT interface validators (e.g. raising ForbiddenAttribute because you spelled 'displayname' incorrectly)
<jtv> gary_poster: I guess I could try to verify the result of getUtility instead of the component.
<jml> maybe some of those utilities don't have correct implements() lines.
<jtv> Since 93 of the 124 failures I have now are DoesNotImplement, I think I'm verifying the wrong object.
<jtv> jml: wouldn't that be noticed immediately?
<jtv> gary_poster, jml: checked out a first case of DoesNotImplementâ¦ the implements() is in a base class.
<jtv> I suppose that would also apply for all these vocabularies that are giving me the same error.
<jtv> If I check getUtility(interface) instead of utility_registration.component, I only get 11 instances of DoesNotImplement.
<gary_poster> huh
<gary_poster> I have not worked with utility registrations enough to know how unexpected that might be
<gary_poster> I'd say stick with whatever spelling seems to reflect reality and move on, myself.  That might be stating the obvious.
<Ursinha> is is just me or launchpad has performance issues today?
<Ursinha> scripts are timing out
<gary_poster> Ursinha, I haven't experienced any problems myself, but graphs seem to suggest that we had increased db load and an increased number of errors starting about six hours ago.  I have no idea what that might be: we have not had a rollout since the 16th.
 * gary_poster tries to find page performance report, and tries to remember other tools for this.
<Ursinha> gary_poster: I'm not able to generate ubuntu reports for a while; in general it takes about 6 minutes and now it's timing out
<StevenK> cjwatson: Sorry about that, I was picking up my wife and having dinner. Did you find someone?
<gary_poster> StevenK, hey, I did it for him
<StevenK> gary_poster: Excellent, thanks.
<gary_poster> StevenK, do you happen to have an idea of something that might have changed six hours ago to make our db load and errors increase?  It is not an increased number of requests, according to the graphs.
<StevenK> And the recipe fails due to chroot problem? Awesomeness
<gary_poster> (number of requests: https://lpstats.canonical.com/graphs/AppServerRequestLpnet/ ; 5xx s : https://lpstats.canonical.com/graphs/AppServer5XXsLpnet/ ; db load : https://lpstats.canonical.com/graphs/DBCpuLoadAppServers/)
<gary_poster> heh, yeah StevenK. :-/
<StevenK> gary_poster: That isn't your fault, though.
<gary_poster> StevenK, yeah, cjwatson thought it might be puppet
<StevenK> gary_poster: The only thing I can think of that I did six hours ago was finish work. :-)
<cjwatson> lamont is looking at it
<gary_poster> StevenK, that's it! ;-)
<StevenK> Aw, man. Now I have to work the weekend. :-P
<gary_poster> :-)
<StevenK> gary_poster: Just thinking about it, check with the LOSAs, mizuho and cocobanana were brought under the iron fist of puppet around that time
<gary_poster> ok thanks StevenK, I'll ask them (do we even have any?)
<gary_poster> I'll find out :-)
<Daviey> Ursinha: yeah, i am seeing timeouts aswell.
<Ursinha> Daviey: all of our scripts are failing, desktop team as well
<jtv> gary_poster: any feedback on lp:~jtv/launchpad/audit-utilities would be most welcome.  Run utilities/audit-utilities.py to see what I get so far.
<jtv> Not now though; I'll be off!
<gary_poster> jtv, cool.  working on emergency; please ask Monday and will be happy to
<gary_poster> Have a great weekend
<jtv> same to you!
<stub> Cool. fiera is doing recursive queries now. Didn't realize anyone was using them yet.
<bigjools> Oo
<deryck> henninge, ping for standup
<henninge> oh, yeah
 * henninge boots phone, looks for headset
<jelmer> Hmm, qastaging is giving me a "Internal Server Error" for loggerhead - is there an easy way to see the actual error?
<henninge> abentley: Hi!
<abentley> henninge: hi!
<henninge> abentley: can we mumble about iorecorder?
<abentley> henninge: 1 sec
<cjwatson> Does anyone know how to make https://code.launchpad.net/~launchpad/+archive/ppa/+recipebuild/73739 (from https://code.launchpad.net/~launchpad/+recipe/meta-lp-deps-on-demand) try again?
<cjwatson> lamont tells me that the buildd problem is fixed now
<cjwatson> maybe people in the ~launchpad team get more buttons there than I dod
<cjwatson> *do
<bigjools> looks like the build page has a bug
<bigjools> says it's finished but offers me a link to cancel it still
<cjwatson> me too
<cjwatson> maybe if we cancel it will try again later?
<bigjools> mebbe
<bigjools> don't know much about this stuff
<cjwatson> I can't rescore it either
<cjwatson> who does?
<bigjools> I can :)
<bigjools> abentley does
<cjwatson> I get the link, but it says "Cannot rescore this build because it is not queued."
<bigjools> huh
<bigjools> yes the page is buggy
<bigjools> I bet it thinks "chroot problem" is not a real error
<abentley> cjwatson: you request a build from https://code.launchpad.net/~launchpad/+recipe/meta-lp-deps-on-demand
<cjwatson> abentley: I'm not sure I have the right team access to request a build that goes into ~launchpad/+archive/ppa
<cjwatson> (if I did, that would probably be a bug)
<cjwatson> abentley: can you?
<abentley> cjwatson: I see the "request build(s)" link on the page, so I assume I do.
<abentley> cjwatson: Do you not see it?
<cjwatson> I do, but surely I only have access to request builds into PPAs owned by myself or teams I'm a member of; I am not a member of launchpad
<bigjools> cjwatson: I'll request a build, is it just lucid you need?
<cjwatson> yeah, the others look done
<bigjools> done
<cjwatson> thanks
<allenap> adeuring, bac: Have you got time for a short-ish review? https://code.launchpad.net/~allenap/launchpad/librarian-log-bug-828151/+merge/72097
<bac> allenap: i do
<allenap> bac: Thanks :)
<bac> np
<deryck> bigjools, there's an lp question about sudo error and busted builds I'm working through....
<bigjools> deryck: should be fixed
<deryck> bigjools, ok, thanks.
<bigjools> :)
<deryck> easy peasy :)
<bac> allenap: done
<allenap> Thanks bac.
<deryck> sinzui, ping.
<sinzui> hi deryck
<deryck> sinzui, hi there.  Do you think you could spare a minute to help kate in:  https://answers.launchpad.net/launchpad/+question/168164 ?
<deryck> sinzui, you could probably directly answer her as quickly as walk me through how to answer her :)
<sinzui> This might be a permission issue. Lots of attr are only setable by LOSAs because Ubuntu one made 100s of members project owners
<sinzui> investigates
<deryck> sinzui, thanks!
<sinzui> s/Ubuntu one/Ubuntu once/
<deryck> sinzui, can I assign the question to you and note that you're looking into it?
<deryck> sinzui, ah, never mind.  I see you did assign it.  Sorry.
<deryck> stupid page reloads.
<sinzui> deryck, datereleased is not directly setable. it is implicitly set by the Web UI when the series status is set to current.
<sinzui> deryck, since the releases are being done by API, there is no way that the date is set. the field is not exposed in the UI either
<deryck> ah, ok.
<sinzui> deryck, I think we need to choose a) push the rule into the model so that the status attr also updates the datereleased attr or b) expose the datereleased attr over the api to let owners change it anytime they like.,
<deryck> sinzui, I like b personally.  Makes sense to me an owner might want to set this themselves, if doing everything else over the API.
<sinzui> I see datereleased is exported over the API. I think someone can set it
<sinzui> deryck, the rule is in place to ensure the field is not set in an impossible status. field validator is needed
<jcsackett> sinzui: mumble?
<deryck> sinzui, can't the validator kick in, even if set over the API?  But it sounds like this is possible anyway, if it's exposed.
<sinzui> ah, the field is editable over the api by anyone with LP moderate, which hopefully includes the series RM
<jcsackett> oh, sorry, i see you're in the middle of a conversation. :-P
<sinzui> deryck, the validator will kickin if we define a new field or update the existing date field to use constraint=???
<cjwatson> StevenK: do you think you could run dpkg-xz-support through again, once you see this?  I'd like to establish whether it has anything to do with the launchpad-dependencies breakage (which seems not unlikely) before doing further manual investigation
<deryck> sinzui, ah, I was thinking of the old bugs way of having a mutator in the model that does the validation in Python. This doesn't work if there are db constraints doing the validation now.
<sinzui> deryck, The answer to the question is the jeremy is not an admin or registry expert. I think we want to change the security checker so that the series.driver (RM) or all ubuntu drivers can change (version name status nominatedarchindep changeslist datereleased)
<sinzui> I will update the question and link it to the really old bug we have been trying to fix by changing the Ubuntu team structure
<deryck> sinzui, ok, cool.  Thanks much!
* adeuring changed the topic of #launchpad-dev to: https:dev.launchpad.net/ | On call reviewer: bac | Critical bugs: 238 - 0:[#######=]:256
<sinzui> jcsackett, I sorted out the distro stuff. I can talk now
<jcsackett> sinzui: i just grabbed lunch. ping you when i'm done?
<sinzui> yes
<jcsackett> sinzui: i'm getting on to mumble now.
<henninge> help!
<henninge> I am totally stuck in trying to get a js test to run.
<henninge> Something to do with loading, I think
<henninge> hm, just solved one thing but there was more ...
<henninge> Which js file do I need to load to have the "LP" object defined?
<jml> henninge: lib/lp/app/templates/base-layout-macros.pt?
<henninge> oh
 * henninge looks
<jml> it's just a grep-informed guess.
<henninge> I only searched js files.
<henninge> jml: yeah, it's there
<henninge> but that's no use for a test, so I will have to create a dummy I guess.
<jml> henninge: grep also points me at lib/lp/code/javascript/tests/test_branchrevisionexpander.html:
<jml> henninge: which, at a guess, declares a dummy.
<henninge> yes it does
<henninge> jml: thanks
<jml> henninge: don't thank me, thank 'bzr grep "var LP"'
<henninge> ;-)
<deryck> henninge, do you want LPS object or the js web service client?
<henninge> deryck: LP.cache
<henninge> but I got it figured out now.
<deryck> henninge, ok.
<flacoste> sinzui: you might interested in bug 829678
<_mup_> Bug #829678: Person picker widget to add a member says it fails but really didn't <person-picker> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/829678 >
<sinzui> flacoste, no I am not. It is a dupe of the bug that jcsackett is working on
<flacoste> sinzui: a cool, then!
<sinzui> I will update the master bug because I reported nothing happen because the spinner failed.
<benji> jam: is there a particular person I should ask to review a small bzr MP?  (https://code.launchpad.net/~benji/bzr/bug-702914/+merge/72239)
<benji> darn, I bet jam is quite AFK at the moment
<jelmer> benji___: hi
<benji> hi jelmer
<jelmer> benji___: it's really nice to see tests for the lazy import threading issue
<jelmer> benji: have you seen bug 396819
<jelmer> ?
<_mup_> Bug #396819: AttributeError: _factory - lazy imports are not threadsafe <lazy-imports> <oops> <Bazaar:Confirmed> <loggerhead:Triaged> < https://launchpad.net/bugs/396819 >
<benji> jelmer: nope, I hadn't seen that; it looks like a dupe of the other (or rather, the other is a dupe of 396819)
<jelmer> benji: yeah, indeed
<abentley> bac: could you please review https://code.launchpad.net/~abentley/launchpad/upgrade-not-branch-error/+merge/72242 ?
<bac> abentley: sure
<abentley> bac: thanks.
<benji> jelmer: do you know if the bzr guys look at their review queue regularly (i.e., will they find my MP lying there and pick it up) or should I email one or more of them?
<abentley> benji: they have one guy assigned to the review queue, called the "patch pilot", every week.
<bac> looks good abentley
<benji> abentley: great, thanks
<abentley> benji: from their topic, looks like it's Riddell this week;.
<abentley> bac: thanks.
<benji> abentley: cool, I'll anticipate hearing from him then
<jelmer> benji: I was actually just doing a review :)
<benji> jelmer: yay!
<jelmer> benji: I'd like to get some input from jam too though, as he's probably most familiar with lazy_import and all its corner cases.
<benji> makes sense
<jelmer> benji: do you know if there's any way to find out why loggerhead on qastaging is failing? It's just giving me a "Internal Server Error" without referencing an OOPS or anything like that.
<benji> jelmer: not off the top of my head... but I'll look at it in a minute and see if I can think of anything
<jelmer> benji, thanks
<nigelb> what's the correct alternative to EDGE_SERVICE_ROOT these days?
<abentley> nigelb: production.
<abentley> nigelb: i.e. LPNET_SERVICE_ROOT
<nigelb> aha
<nigelb> abentley: thanks!
<abentley> nigelb: no problem.
<benji> jelmer: it looks like it's having an error trying to report an error that it's having: https://pastebin.canonical.com/51493/
<henninge> abentley: maybe you'd enjoy reviewing this? https://code.launchpad.net/~henninge/launchpad/bug-824435-failure-reporting/+merge/72255
<henninge> abentley: but I can also ask the OCR
<abentley> henninge: I'm happy to look at it.
<henninge> abentley: thanks
<nigelb> flacoste: aww, I wish I could see those graphs
<abentley> henninge: in some browsers (IE), the last element in a list must not have a comma following it.  Please fix that before landing.
<abentley> henninge: It also looks like you've made changes that will break the existing use of IORecorder, i.e. renaming success -> response.
<henninge> abentley: I added "respond" and made "success" use that.
<henninge> but  I just found something else, so expect another revision.
<henninge> abentley: I also ran test_lp_client with that new iorecorder.
<abentley> henninge: okay, cool.  I didn't see the new success.
<flacoste> nigelb: i can't give you access, but I can copy them in a public location
<nigelb> flacoste: that would work, thanks!
<jelmer> benji: sorry, was away for a bit for dinner
<jelmer> benji: that error looks odd
<jelmer> benji, out of curiosity, where did you look to find that log?
<benji> jelmer: note that I'm not sure that that error is the result of requests, but it's the only error in the logs
<benji> jelmer: from /srv/launchpad.net-logs/qastaging/asuka/launchpad.log on devpad.canonical.com
<jelmer> benji: ah, thanks
<henninge> abentley: screenshots in the MP! ;-)
<abentley> henninge: cool.
<henninge> abentley: I also pushed a new revision
<abentley> henninge: r=me
<henninge> abentley: thanks! ;-)
<nigelb> I forget, how is it that I run only one test?
<flacoste> nigelb: http://people.canonical.com/~flacoste/LPProjectCriticalClosed-2011-04-01_2011-0820.png
<flacoste> and http://people.canonical.com/~flacoste/LPProjectCriticalFiled-2011-04-01_2011-0820.png
<nigelb> flacoste: yay, thanks!
* bac changed the topic of #launchpad-dev to: https:dev.launchpad.net/ | On call reviewer: - | Critical bugs: 238 - 0:[#######=]:256
<deryck> Have a nice weekend, everyone.
<bac> sinzui_: you still around?
<sinzui_> I am
<sinzui> irc was lying
<bac> sinzui_: mvo is reporting he cannot see private bugs for a source package he is the maintainer
<bac> has anything changed around private bug visibility?
<sinzui> No
<sinzui> You still need to be subscribed
<bac> he's also indirectly in the team that is the bug supervisor for the distro
<sinzui> but is the supervisor subscribed to those bugs?
<sinzui> marking a bug private does not subscribe the supervisor, though I hope to fix the 6 year old bug in 4 weeks
 * bac looks at userCanView
<sinzui> bac: which distro? Ubuntu?
<bac> yes
<bac> update-manager package
<bac> mvo
<bac> sinzui: i thought the bug supervisor was subscribed to bugs when they are made private
<nigelb> bac: I just tested on qastaging. Only security supervisor is sub'd for private bugs.
<sinzui> bac: Consider the changes made by structural subscriptions. User can be very specific about what they subscribe to and about the team that is subscribed. When a bug is made private, the structural subscribers are converted to bug subscribers. In the past, the bug supervisor could not control which email he got...he was subscribed to every bug, so he was always in the set of people subscribed. Now that the team can filter
<sinzui>  the notifications, the team is implicitly opting out of private bug subscriptions
<bac> oi
<sinzui> There is only one code point where a bug supervisor is explicitly subscribed to a bug, and it was written by statik 4 years ago. When a bug is created for a project with private bugs, the supervisor is subscribed
<bac> ah, right, that's why a project must have a bug supervisor before 'private by default' is enabled. that bit is what was confusing me.
<sinzui> bac, Since I can see mvo's +participation page, I assume his membership in ~ubuntu-bugs is correct. Can other members of that team see the bugs? Are then only subscribed via that team
<sinzui> bac: there is a small chance that something else private on that page could cause a 403 oops for him as we have seen on branch pages
<nigelb> anyone around?
<nigelb> gah, monday then.
#launchpad-dev 2011-08-20
<StevenK> cjwatson: I've tossed your branch back at ec2.
<StevenK> It has been running for an hour.
<cjwatson> StevenK: thanks!
<nigelb> Hey, how do I get the '/bug/1234' for a bug I just made with factory?
<nigelb> just manually create it?
<cjwatson> StevenK: argh.  OK, I'll debug it some more on Monday ...
<cjwatson> one step forward, two steps back
<jelmer> StevenK: argh, looks like raced and you won :-/
<jelmer> *we
<StevenK> jelmer: Hopefully I solved it the same way you did
<kb9vqf> How would I add "3.0 (quilt)" as an accepted source format for a distribution on my private Launchpad installation?
<kb9vqf> I can't seem to find the controlling table
<kb9vqf> wgrant_: Are you around?
<kb9vqf> How would I add "3.0 (quilt)" as an accepted source format for a distribution on my private Launchpad installation?
<kb9vqf> I can't seem to find the controlling table :)
<kb9vqf> Never mind, I figured it out after grepping through the Launchpad archive processor
<kb9vqf> The numeric values in the sourceformatselection table control the allowed upload types
<kb9vqf> 0 is native 1.0
<kb9vqf> 1 and 2 are the new quilt 3.0 and native 3.0 formats
<kb9vqf> in case anyone else runs into this problem :)
#launchpad-dev 2011-08-21
<nigelb> lifeless / wgrant: around longer?
<wgrant> nigelb: Hi.
<nigelb> wgrant: just need a quick help with a test
<wgrant> nigelb: Sure.
<nigelb> I add some code to the linkchecker
<nigelb> and I'm writing tests for private bugs
<nigelb> Is thre a place I can see an example of a private bug creation?
<wgrant> Something like factory.makeBug(private=True) doesn't do what you want?
<nigelb> sure, but I also want a use who can see that :)
<wgrant> makePerson()
<nigelb> I also want a user who can see the bug :)
<wgrant> bug.subscribe(person, person)
<nigelb> ahh~
<nigelb> also, the '/bug/1234', I'll need to just build that myself right?
<wgrant> But the anonymous user won't be able to call bug.subscribe(); you might need to use 'with celebrity_logged_in("admin")' or so.
<wgrant> canonical_url(bug) will give you the URL.
<nigelb> trying now
<nigelb> Also, urgh @ YUI tests.
<nigelb> wgrant: er, lol. what's the guranteed way to come up with an invalid bug number?
<wgrant> nigelb: Well, depending on the test, you could find the max bug number and add 1. Or deliberately make a hole in the postgres sequence. Or use sys.maxint. Or something.
<nigelb> ah, there'sa wway to find max bug number.
<lifeless> nigelb: why do you want an invalid bug number ?
<nigelb> lifeless: the link checker returns valudes when the bug number is invalid.
<nigelb> (or private)
<nigelb> I want to catch both cases.
<lifeless> 'valudes' ?
<nigelb> *values
<lifeless> it is doing that? or you think its doing that ?
<nigelb> It is doing that. I wrote the code ^-^
<lifeless> If you used the approach I suggested, you don't need to discriminate between private (won't be found in a search), nonexisting (won't be found in a search)
<nigelb> Ah
<nigelb> So, just give it a private bug and then assume the same for invalid?
<lifeless> unless your code actually discriminates between private and invalid, any change in behaviour is out of the scope of your code and you shouldn't test it there
<lifeless> does your code discriminate ?
<nigelb> nope
<lifeless> then you should rely on the interface you're using (bug search) and its tests ;)
<nigelb> oh.
<nigelb> but I still should be testing my code for valid vs invalid bugs right?
<nigelb> where invalid maybe be private or nonexistant.
<nigelb> But you're saying that I should not test for private and nonexistant since my code doesn't differentiate between them
 * nigelb is unwell and not completely awake.
<nigelb> Ugh, I feel like stabbing YUI.
<lifeless> nigelb: thats what I'm saying, yes.
<lifeless> a unit test should only test the behaviour of the unit under test
<lifeless> *integration* tests may test multiple layers, but they should be few and far between.
<nigelb> lifeless: I did what you suggested :)
<nigelb> Now, onto testing the javascript code
<nigelb> aka stabing YUI.
<nigelb> hm, can't it working. Maybe I should ask someone on Monday.
 * jelmer waves
<StevenK> jelmer: O hai -- could you look at the db-devel failure and see if the conflict I fixed had anything to do with it?
<jelmer> StevenK: Sure, let me have a quick look
<jelmer> StevenK: looks like they have somewhat to do with your branch - db-devel has bzr 2.4, which has slightly different locking semantics
<jelmer> argh, I wish testr integrated better with the launchpad testsuite :-/
<jelmer> that said, I can't reproduce the errors here, they look spurious
<StevenK> jelmer: Feel free to force a build, then
<jelmer> StevenK: it haz been done
<nigelb> 54
<nigelb> gah
<lifeless> *yawn* morning
<lifeless> jelmer: please file bugs about things that make the integration poor
<jelmer> lifeless: will do
<lifeless> wgrant and I are using it for lxc based parallel test dispatch
<jelmer> ah, neat
<lifeless> and its new project time again
<lifeless> \o/
<jelmer> lifeless: what are you starting now?
<lifeless> jelmer: gpgfixtures
<lifeless> jelmer: test support for gpg - extracting the stub keyserver etc from lp
<lifeless> so that I can test the gpg verification web service sensibly without code duplication or importing LP
<lifeless> jelmer: I made 3 new LP projects last week :P
<jelmer> lifeless:
<jelmer> whoa
<lifeless> lp:python-oops, lp:python-oops-wsgi and lp:python-oops-datedir-repo
<lifeless> jelmer: https://dev.launchpad.net/CreatingNewProjects has probably been polished since you last read it
<jelmer> lifeless: None < any(last_modified_time) :P
<jelmer> lifeless: does that mean things that are split off no longer have to be "lazr" projects?
<jelmer> that's great
<lifeless> https://launchpad.net/python-gpgfixtures
<lifeless> jelmer: care to do a real simple review ? :)
<lifeless> https://code.launchpad.net/~lifeless/launchpad/gpgme-egg/+merge/72366
<jelmer> lifeless: ooh
<jelmer> lifeless: I didn't realise 0.2 was out
<jelmer> lifeless: +1
<lifeless> thanks
<lifeless> ok
<lifeless> -not- what you want to find in every codepath using a test server:
<lifeless>         sleep(0.02)
 * wallyworld_ is off to visit the bank armed with a baseball bat
<lifeless> wallyworld_: ?!
<wgrant> 926	+            cls, 'PROJECT', 'Product',
<wgrant> 927	+            'Display search results associated with products')
<wgrant> :(
<wgrant> 937	+            cls, 'PROJECTGROUP', 'Project',
<wgrant> 938	+            'Display search results associated with projects')
<wgrant> :(
<lifeless> isn't that wrong ?
<wgrant> Yes.
<mwhudson> "Though there is work going on to demoralise the data to make the DSP queries fast"
<mwhudson> best typo evah
#launchpad-dev 2012-08-13
<lifeless> \o/
<lifeless> 10000  OOPS-fb8c41fc98e9a541a1175a570450f288  Unknown
<StevenK> wgrant, wallyworld_: https://code.launchpad.net/~stevenk/launchpad/auditor-on-packageupload-reject/+merge/119291
<lifeless> StevenK: how goes auditor ?
<StevenK> lifeless: Still wondering what the next step is.
<lifeless> go on
<StevenK> lifeless: So we have auditor, its fixture, its client and now two callsites in LP
<lifeless> is there a staging auditord yet ?
<StevenK> Nope
<StevenK> lifeless: So I guess we need to plan to deploy auditor for staging/qas and then slap it into an RT.
<lifeless> yes.
<lifeless> this might be a good time to do the refactoring I blathered on about
<StevenK> Oh? :-)
<lifeless> voice?
<StevenK> Skynet? Mumble?
<lifeless> the robots are coming!
<lifeless> (skype)
<StevenK> Right, let me get that set up
<StevenK> lifeless: Apparently, you're red
<StevenK> lifeless: Firebug is making Firefox do the crash-restart-crash loop
<nigelb> StevenK: use in-built dev tools
<StevenK> Oh, is that what Mozilla is trying to tell me?
<StevenK> And now it's lost my session details
<StevenK> I guess it got too embarrased
<nigelb> lol
<StevenK> Maybe I'll just switch to Chrome
 * lifeless thought firebug *was* the built in dev tool
<StevenK> Haha
<nigelb> There's been built-in dev tools for ages in Firefox now. And it's been getting really good.
<nigelb> Whoosh. Missed the irony.
<micahg> StevenK: bug 1025011
<_mup_> Bug #1025011: Firebug extension causes firefox to crash (can be triggered by opening HUD) <verification-done> <verification-needed> <Mozilla Firefox:Confirmed> <Global menubar extension:Fix Committed> <firefox (Ubuntu):Fix Released> <firefox (Ubuntu Natty):Fix Committed> <firefox (Ubuntu Oneiric):Fix Committed> <firefox (Ubuntu Precise):Fix Committed> <firefox (Ubuntu Quantal):Fix Released> < https://launchpad.net/bugs/1025011 >
<StevenK> Handy
<micahg> so basically, Chris fixed it, but the next firebug update discovered another latent bug in the code
<StevenK> wgrant: I don't understand the remaining 5 failures. :-(
<micahg> StevenK: so, either use 1.14.0 with Firefox 14 in the release pocket, 1.14.1 with firefox in -proposed, 1.14.2 and you're on your own
<wgrant> StevenK: 'sup?
<wgrant> StevenK: This is the testbrowser upgrade?
<StevenK> micahg: Or switch to Chromium in anger.
<StevenK> Which I did.
<StevenK> wgrant: Yah.
<micahg> StevenK: and be open to lots of CVEs, have fun :)
<wgrant> StevenK: What are the failures?
<StevenK> wgrant: http://pastebin.ubuntu.com/1144330/
<StevenK> I've fixed the third failure in xx-person-subscriptions.txt, but I'm not sure about the first two.
<wgrant> StevenK: goBack
<StevenK> wgrant: Which is only in two of them
<wgrant> :(
<wgrant> goForward? :)
<StevenK> None of them :-P
<wgrant> logging-in.txt is pretty obvious
<lifeless> micahg: which code is at fault ?
<micahg> lifeless: globalmenu-extension
<wgrant> xx-person-subscriptions.txt is odd
<StevenK> Hm, Chromium doesn't want / for search :-(
<lifeless> micahg: ah well, we knew it was hairy.
<micahg> that's the other option, disable the globalmenu :)
<wgrant> StevenK: Oh
<wgrant> StevenK: Try to think through the xx-person-subscriptions.txt failure
<wgrant> The first one, that is
<StevenK> wgrant: I'm still trying to work out why you say logging-in.txt is obvious
<lifeless> wow
<lifeless> postgresql schema patch downtime -> the dev wiki page on live patching is 5th in google.
<wgrant> StevenK: It's not using normal browser methods
<wgrant> StevenK: It uses some OpenID thing
<wgrant> lifeless: Indeed, we seem to have #5 and #6, from a couple of hopefully unbiased browsers.
<wgrant> StevenK: So, the first xx-person-subscriptions.txt failure is because it's now sending a sensible Referer
<wgrant> StevenK: The view uses Referer to set cancel_url
<wgrant> So it now ends up on the right page
<wgrant> Not the one it expects.
<StevenK> Which I thought I removed in anger
<lifeless> http://postgresql.1045698.n5.nabble.com/ChronicDB-Live-database-schema-updates-with-zero-downtime-td1901036.html
<lifeless> also lol - 'ChronicDB is currently a private technology'
<lifeless> moving right along
<lifeless> have we done a second schema patch without slony now ?
<wgrant> lifeless: No
<wgrant> lifeless: Hopefully Colin will have one for us tomorrow.
<StevenK> wgrant: logging-in.txt is making use of testbrowser to do stuff, it just uses the extra methods to parse what came back
<lifeless> anyone remember the statistic from flacoste about velocity improvement from FDT ?
<wgrant> StevenK: It passes the URL into consumer.complete, which looks a bit sus to me.
<StevenK> wgrant: It does? Where?
<wgrant> StevenK: In complete_from_browser
<StevenK> Ah
<StevenK> wgrant: So I guess xx-person-subscriptions.txt is the test expecting behaviour which is now fixed
<wgrant> StevenK: The first failure, yes.
<wgrant> The others I haven't looked at.
<StevenK> wgrant: How does goBack break the other two?
<wgrant> StevenK: I don't know.
<wgrant> I don't even know if it does.
<wgrant> But it seems very likely.
 * StevenK sigh
<StevenK> I broke buildbot
<wgrant> That's not very nice of you.
<wgrant> Oh
<wgrant> I assumed that was local breakage
<wgrant> My build just failed the same way
<StevenK> I'm just about to land a testfix
<StevenK> As soon as bzr push completes
<wallyworld_> StevenK: wgrant: either of you guys adding the new Embargoed information type to the enum? i'll do it if no-one else is
<wgrant> I'm not.
<wgrant> Not sure it's worth adding yet, but maybe :)
<StevenK> I'm swearing at testbrowser and buildbot
<wallyworld_> i figured we'd need it done first so things could use it
<wgrant> Well
<wgrant> We should be sure it's going to work first
<StevenK> We are still discussing it, so I'm not sure it's worth leaping yet
<wgrant> I'm currently finishing off sharing policies for Proprietary.
<wgrant> StevenK: Right.
<wallyworld_> ok, i'll hold off. i thought we had agreed to do it
<StevenK> wallyworld_: Do you also do "Ready, fire, aim" ?
<wallyworld_> huh?
<StevenK> As opposed to 'Ready, aim, fire'
<wallyworld_> i must be slow today, don't get the reference?
<lifeless> its slightly whacky humour
<lifeless> the instructions called out by WWI officers - ready (guns loaded etc), aim - aim at the folk running across no mans land, fire - pull the trigger.
<lifeless> StevenK is suggesting that you are doing (firing) before aiming (finishing the discussion).
<lifeless> Personally, I think its a little harsh.
<wallyworld_> especially since i thought we had converged on a solution, but anyway
<wgrant> wallyworld_: We hopefully have, but eg. mrevell was still unclear about it on Friday
<wgrant> And it probably needs to be confirmed with PES
<wgrant> The solution evolved on the Friday call, and Curtis was away on his Friday, so it hasn't had much chance to settle.
<lifeless> should I put my nose in a little?
<wgrant> No.
<wallyworld_> if you want
<wallyworld_> wgrant: there was a card on the board in the next column, hence my thinking we had agreed to it also
<wallyworld_> if it's not final, we shouldn't put the card up yet :-)
<wgrant> wallyworld_: Bah
<wgrant> True
<wgrant> I put that there :)
<wgrant> It's agreed, but not final.
<wallyworld_> np, glad we could clarify
<wgrant> And mrevell appears concerned
<wallyworld_> oh? is there an email i missed?
 * wallyworld_ has to go to soccer, back later
<wgrant> Well, just the steps to sharing beta one. It seems nobody outside the team is clear on what's happening.
<wgrant> we'll see over the next couple of days, I expect
 * StevenK stabs Unity
<StevenK> Firefox crashes, and brings up a crash dialog, so you click Restart and then Unity loses track of the main Firefox window until you fiddle with the dock
<adeuring> good morning
<StevenK> lifeless: You missed a failure -- we tickled a bug in Slony when we dropped a column, so the sequence was killed and Slony got very unhappy with the world.
<StevenK> lifeless: Unless that's what you meant with two tables at once? We didn't drop the tables, we dropped their id columns.
<wgrant> I believe that's the failure to which he referred in his reply
<wgrant> Although misstating it, as you say.
<lifeless> StevenK: bah, you are right.
<StevenK> lifeless: Now you can edit your blog post and thank me. Or something.
<wgrant> Oh, there's a blog post?
<wgrant> Ah, there
<czajkowski> where..
<wgrant> http://rbtcollins.wordpress.com/2012/08/13/minimising-downtime-for-schema-changes-with-postgresql/
<czajkowski> ahh
<lifeless> StevenK: or I can let it stand wrongly vague :P
<StevenK> lifeless: "Or something."
<lifeless> :)
<czajkowski> lifeless: StevenK wgrant anyone any thoughts on https://lists.launchpad.net/launchpad-dev/msg09565.html
<StevenK> Yes. None are printable.
<cjwatson> wgrant: Thanks.
<lifeless> czajkowski: blueprints are for summit basically
<lifeless> czajkowski: so 'just for summit' is sufficient as an answer
<lifeless> czajkowski: though there may be a deeper question that isn't clear yet
<lifeless> StevenK: wgrant: so we dropped the id column from one or two tables ?
<czajkowski> lifeless: I suspect cjohnston is working on summit stuff and wants to make a change, but hasn't explained what besides wording
<lifeless> czajkowski: I share your suspicion.
<czajkowski> lifeless: which doesnt help get him an answer either :/
<lifeless> czajkowski: and thus noone has answered ;)
<ev> Presumably I should be using ETags and gzip when communicating with Launchpad from another webservice, right? Anyone know if a thread-safe alternative to httplib2 for that?
<lifeless> httplib2 is threadsafe with the patch jml did, we think.
<ev> oh awesome
<czajkowski> lifeless: true, I'm just following up on stuff from my week off
<lifeless> s/thread/concurrency/
<jml> ISTR someone pushing for a release and getting it into 12.04
<cjwatson> I was asking for one
<cjwatson> But wasn't that lazr.restfulclient not httplib2?
<wgrant> lifeless: We dropped an ID column from two tables.
<lifeless> wgrant: replacing it with some natural PK ?
<lifeless> s/it/them/
<wgrant> lifeless: Right
<wgrant> Dropping a serial column automatically drops the underlying sequence
<wgrant> Slony automatically polls sequence values
<wgrant> => slony broke
<wgrant> The only ID columns we've dropped in recent history are BranchRevision.id and OAuthNonce.id
<wgrant> And the latter wasn't replicated
<wgrant> and I don't remember when we did BR.id
<wgrant> So it's possible that this isn't a regression
<lifeless> fark HUD.
<lifeless> DIAF.
<wgrant> What's it doing now?
<lifeless> killing FF
<wgrant> Ah
<wgrant> Disable the globalmenu extension? :)
 * cjwatson looks at database/schema/comments.sql and wonders how he's supposed to "keep [it] alphabetical by table"
<wgrant> lalalalal
<cjwatson> Should the first part of my branch be to sort it? :-P
<wgrant> Well
<wgrant> comments.sql is sort of dead to us now
<cjwatson> https://dev.launchpad.net/PolicyAndProcess/DatabaseSchemaChangesProcess
<cjwatson> Just saying
<wgrant> Whether it should continue to exist at all is not exactly clear
<cjwatson> "If you have removed or renamed a table or column, update database/schema/comments.sql."
<lifeless> so, comments.sql is no longer applied in production.
<wgrant> Right, comments.sql needs updating in that case or make schema will fail
<wgrant> It doesn't say anything about adding new ones :)
<lifeless> We need to move:
<lifeless>  - comments.sql
<lifeless>  - fti
<lifeless>  - sampledata
<cjwatson> wgrant: "Comments on new tables and columns need to be added to database/schema/comments.sql. "
<lifeless> into the lazr.postgresql patch system
<wgrant> Curses.
<lifeless> and drop the LP one entirely.
<lifeless> cjwatson: don't stress on the alphabetical, please do add comments until we transition.
<cjwatson> In the short term should I just put COMMENT statements in the patch itself?
<lifeless> no, comments.sql.
<lifeless> it won't apply in prod, but we don't care.
<wgrant> There was some discussion a while back about moving them to patches
<wgrant> As we did with trusted.sql
<wgrant> Have you dropped that idea?
<lifeless> wgrant: not at all, just we haven't actually done it AFAICR, and unlike trusted.sql its not functional.
<wgrant> Yeah
<lifeless> StevenK: thanks for pointing that out. Updated.
<mgz> what is this a symptom of does anyone know? can't get launchpad.dev to respond this morning...
<mgz> 2012-08-13T09:55:34 INFO root Startup time: 9.164 sec real, 9.110 sec CPU
<mgz> channel 3: open failed: connect failed: Connection refused
<mgz> channel 4: open failed: connect failed: Connection refused
<lifeless> could be rabbitmq not running ?
<cjwatson> That's an error message from ssh
<cjwatson> If that helps
<mgz> ah, so I borked my ssh forwarding perhaps
<mgz> will also poke the rabbit.
<mgz> okay, so it seems nothing is actually listening on 8080, zope.server.http came up on 8085, not sure if that's normal, haven't got a log from when it was working earlier to compare against
<mgz> ...or have I
<mgz> hm, did that last time as well.
<mgz> but 8080 is what used to work...
 * mgz looks at apache setup
<wgrant> mgz: The normal launchpad.dev Apache config listens on 80
<wgrant> (and 443)
<wgrant> 8080 is usually unused
<mgz> well, apache is not running... >_<
<mgz> what changed...
<mgz> manually starting it and all is well
<mgz> so close to not having to interveen, but need manual fixup too often, half the time it's me being dumb but this is too much of a pain
<mgz> so, I think apache doesn't get added to the things that should be run on startup perhaps? install, use launchpad is fine, didn't come back after reboot...
<cjwatson> lifeless: https://code.launchpad.net/~cjwatson/launchpad/queue-filter-source/+merge/119225 - I'd be happy to try running this on dogfood.  Is there a sensible way to extract a compiled version of the query there (some kind of profile mode or something) or do I have to work it out by hand?
<wgrant> cjwatson: bzr merge lp:~cjwatson/launchpad/queue-filter-source
<wgrant> make initscript-stop initscript-start
<wgrant> Browse :)
<wgrant> Ah, but you won't get in-page query logs unless you're in ~launchpad...
<cjwatson> And look at dogfood-nohup.out or something?
<wgrant> You could change the feature flag to allow you too :)
<cjwatson> Oh right.  I could add myself to ~launchpad on DF :P
<wgrant> The visible_render_time flag shows a partial query log at the end of the page
<wgrant> Or that, yes
<cjwatson> How about API queries?
<wgrant> Good luck with that :)
<wgrant> Can't even use ++oops++ with the API
<wgrant> Perhaps use LP_DEBUG_SQL=1 bin/harness?
<cjwatson> Actually I guess DistroSeries:+queue lets me do basically the same query, so not necessary in this case
<wgrant> Yeah
<wgrant> That's what I assumed you'd test
<Ergo^> morning
<czajkowski> blue - jam jelmer mgz vila https://bugs.launchpad.net/launchpad/+bug/1035338
<_mup_> Bug #1035338: An unexpected error has occurred while updating a Launchpad branch <Launchpad itself:New> < https://launchpad.net/bugs/1035338 >
<mgz> just jam/mgz czajkowski :)
<czajkowski> ah ok
<czajkowski> thanks
<mgz> czajkowski: that oops won't load for me, just spinns during lookup, but have responded on the bug
<Ergo^> morning
<cjwatson> Hmm.  Upgrading dogfood says:
<cjwatson> lazr.config.interfaces.ConfigErrors: ConfigErrors: database does not have a auth_master key.
<cjwatson> database does not have a auth_slave key.
<cjwatson> Something to do with r15782 I guess
<cjwatson> ah, the dogfood config needs to be updated
 * cjwatson will sort it out
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: benji | Firefighting: - | Critical bugs: 4.0*10^2
<jam> mgz: I did eventually get the oops to load, though I did it by going to oops.canonical.com directly and then doing a search from there (still took a while)
<jam> interestingly, the content there is an XMLRPC Fault that contains yet-another OOPS
<jam> xmlrpclib.Fault: <Fault -1: 'OOPS-394216d625e87559cda9d8b92d70b90f'>
<jam> look at that one: https://oops.canonical.com/?oopsid=OOPS-394216d625e87559cda9d8b92d70b90f
<jam> It ends up as RequestExpired.
<jam> I wonder if they hit downtime while he was trying to commit?
<wgrant> jam: No, planned DB downtime does not result in timeouts.
<jam> wgrant: even for bzr codehosting?
<wgrant> The underlying OOPS (the RequestExpired) is from the appservers, not codehosting.
<jam> this is a Request timeout in the Storm layer
<wgrant> But yes, even for codehosting.
<jam> now, it does seem to say that it was taking 7s to get a Feature flag looked up...
<jam> which seems very odd
<wgrant> Note also the 4s lookup for a person by ID
<jam> yeah
<wgrant> It's probably GIL contention or similar.
<wgrant> The appservers are single-threaded, except not.
<jam> Isn't that SQL time?
<jam> Or the timing code has enough python in it to cause some time to be in python-land
<wgrant> It's "SQL time"
<wgrant> But measured by Python
<wgrant> So if the interpreter doesn't run for 4s, it'll appear to have taken 4s longer.
<wgrant> It's possible that it's connection latency or firewall breakage.
<wgrant> But it's very similar to the behaviour we saw more frequently before we went to single-threaded* appservers
<wgrant> (* each instance actually has two threads, one to serve webapp requests and the other to serve xmlrpc-private (this is xmlrpc-private), so there can still be contention)
<wgrant> The only theory we have that does not involve persistent network flakiness is threading contention
<wgrant> So threading contention is the going assumption.
<wgrant> It's supported by the fact that we mostly see these on xmlrpc-private requests
<wgrant> And xmlrpc-private requests are rarely CPU-intensive on the appserver.
<wgrant> (so webapp requests have a smaller chance of being starved)
<wgrant> Now
<wgrant> 7s is pretty huge
<wgrant> Bordering on implausibility, really
<wgrant> But the 4s query afterwards means it wasn't just a momentary disruption
<wgrant> We often just see the slow first query
<cjwatson> benji: Could I have a review of https://code.launchpad.net/~cjwatson/launchpad/report-pcj-oops-2/+merge/119209 ?  I'd like to unstick the chain of work that's part of
<benji> cjwatson: sure, but I'm on an interview call at the moment; it may be an hour
<cjwatson> OK, thanks - I probably need to head to a more congenial network environment anyway
<benji> cjwatson: review done
<rick_h_> deryck: left the file as a .odg file and uploaded with images/set up.
<rick_h_> deryck: let me know if you don't see it in the shared U1 folder
<deryck> rick_h_, ok, will check here shortly
<cjwatson> benji: thanks
<rick_h_> congrats frankban
<frankban> thanks rick_h_
<mgz> how do I set a feature flag to default to on?
<mgz> doc says should always use "enabled", but the current state is that enabled should be True
<sinzui> mgz: feature flags are not truly bools
<sinzui> We set the scope to "default" and add the string "on" to indicate we want everyone to get the feature
<mgz> but the codebase must have some idea of which ones are on at present
<sinzui> mgz we remove "on" to disable it. Any string will evaluate to True in that position
<mgz> or all tests, and the testing config, set flags as needed?
<sinzui> sure, we have a page for that
<sinzui> https://launchpad.net/+feature-rules
<sinzui> mgz: tests must set flags to use them. They are off in the test suite
<mgz> ...so landing this will inevitably change the current state till someone sets the new flag on production?
<mgz> and everyone's local test setups will also change?
<mgz> (not that anyone will mind, I'd think, in this case)
<sinzui> mgz: no one setup changes with the addition of a flad, nor does production. That is why we know bugs.dynamic_bug_listings is not finished. I have to enable it to see in in dev, and it must be on in production to see it. People only see the new feature because we enable it for them, or we remove the flags from the code
<mgz> okay, so it's a bit of a hack using a feature flag for this then, but I'll trust that it's the most sensible option
<sinzui> mgz, So reading back to what you first asked...the answer is feature flags cannot be default on because by their nature, they are incomplete.
<mgz> thanks for explaining the details for me sinzui :)
<mgz> hm, seems it's not possible to do boolean logic in a tal expression
<mgz> | relates to path lookup, not evaulation of the objects found
<sinzui> mgz: we use tales adapters to create bools. We also use "|Nothing" to safely adapt to false when an exception is raised
<sinzui> mgz: what path do you want to know is true or false?
<mgz> ...darn disconnects >_<
<mgz> sinzui: basically I want to change not:view/user into not:view/user||not:features/app.root_blog.enabled
<cjwatson> use python:?
<sinzui> No
<mgz> I think the shortest path to that is adding an attribute to the python class that contains that combination
<sinzui> We do not want logic in the templates.
<mgz> python: is bad because it makes everything after it python
<cjwatson> fair enough, I'll shut up :)
<mgz> can't just use it to do the boolean logic correctly on path lookups
<sinzui> mgz, move the logic into the view so that we can write a unit test
<mgz> right, that's pretty much where I'd arrived at. the test I've written is just looking at the html output
<sinzui> mgz: tales makes testing hard, trying to make it support logic ultimately leads to more hours spent trying to test it
<mgz> is there somewhere else I can do this that doesn't involve the databasefunctionallayer?
<sinzui> mgz: I use view = create_initialized_view() then test the bool value of view.show_root_blob
<mgz> I'll grep for some examples, thanks
<sinzui> mgz, given the logic you tried to write, I think you want to add tests to LaunchpadRootIndexViewTestCase
<mgz> that's where I've got 'em
<mgz> still painful for tdd, with multi-second setup time for the layers
<sinzui> mgz, You can try to write the test on the FunctionalLayer, but that wont be must faster
<sinzui> ./bin/test -vvc -t LaunchpadRootIndexViewTestCase lp.app.browser.tests.test_launchpadroot
<sinzui> will be the fastest command
<mgz> oo, and colourful.
<mgz> I've been using -m... apparently that's the same as just giving an arg?
<sinzui> mgz, gary_poster told me no, passing the module path as the last arg is different and faster
<mgz> ehehe
<mgz> okay, this all looks workable
<jml> sinzui:  './bin/test -vvc lp.app.browser.tests.test_launchpadroot  LaunchpadRootIndexViewTestCase' might be faster still
<sinzui> oh
<sinzui> I look forward to faster tests
<mgz> when layer setup takes nearly 10 seconds, it's hard to see a difference...
<sinzui> mgz :(
<mgz> is there any distinction between `with anonymous_logged_in() as user: view = create_initialized_view(..., principle=user)` and just `create_initialized_view(...)`?
<sinzui> mgz yes :(
<sinzui> barry wrote the helper to ignore the user we alreay placed in the interaction with login...
<mgz> if I want to test what someone sees if they arrive at launchpad.net with no cookies, which do I use?
<sinzui> principal is rarely needed though. Generally principal is only needed when we render the view and the page renders the "logged in as" portlet
<mgz> hm. hard to tell what's needed and what's cargo culting when using other tests as templates.
<mgz> just using the minimum that works is tempting, but sometimes that ends up asserting useless things.
<sinzui> mgz, you need to construct the request manually I think. The helper accepts an optional request object (which you may have already setup for the login user. Otherwise, the help will create it's own user and request
<sinzui> mgz, actually, the default request has no cookies, so you do not need to create it
<sinzui> mgz, if the method you wrote is looking at the self.request, pass the logged in user at the principal
<mgz> hm, nope, test method I'm working from just goes getUtility, makePerson, login_person, create_intialized_view, call view to get result
<jcsackett> sinzui: free to chat?
<sinzui> yes
<jcsackett> excellent.
<jcsackett> invite sent.
<sinzui> phone is being odd
<sinzui> It know I got the invite, but says I am not logged in.
<sinzui> something needs a whack upside the head
<sinzui> jcsackett: hold on a moment
<sinzui> Google doubts that I exist
<jcsackett> sinzui: sure. i've cancelled the hangout so i can resend whenever.
<lifeless> cjwatson: you can export LP_SQL_DEBUG (IIRC) to get queries logged to stdout, and then interactively fiddle around till you see the one yo want
<lifeless> cjwatson: or you use ++oops++ if its a regular view to trigger an oops and get timing data; if you're in the right group (~launchpad) you'll get timing data on the initial queries in web views by default
<rick_h_> deryck: ping, how we doing? I'm coming up on EOD in 30 and wanted to check in
<rick_h_> benji: have a sec to look over https://code.launchpad.net/~rharding/launchpad/yuiv5/+merge/119406 sometime?
<rick_h_> lifeless: this is per our conversation, if you know of a better way to do it via makefile magic appreciate feedback/tweaks. ^^
<deryck> rick_h_, on call, just a minute sorry
<rick_h_> deryck: np, knew it would be tough to get time today.
<lifeless> deryck: o/ Did I miss our call ?
<lifeless> deryck: oh no, its later today. Cool.
<rick_h_> lifeless: once I get code review/approved I'll run through ec2 and see what goes boom (hopefully nothing)
<lifeless> rick_h_: so that would be now ;>
<rick_h_> lifeless: ah, missed you approved, just saw the comment
<rick_h_> sinzui: ping, UI question for you.
<sinzui> hi rick_h_
<rick_h_> sinzui: just used the updated bug linking picker and the difference struck me a bit and wondered what you thought
<rick_h_> most pickers have always been a find, click on it to select experience
<rick_h_> when I just did that I ended up with my bug opened in a new tab, had to go back and look and notice a new "Link" button
<sinzui> yes, that is odd
<rick_h_> twisted me up, intentional?
<rick_h_> or should I file a bug on the updated UI?
<sinzui> wallyworld_: has been migrating the old widget to act like a picker, then actually be a picker
<rick_h_> yea, why I wasn't sure what discussions might have gone on before I declare it a 'bug'
<sinzui> I think the selecting the  item as you suggest is correct
<sinzui> rick_h_: I agree we want to fix the behaviour
<rick_h_> sinzui: ok, will file a bug and try to ping wallyworld_ on it in case it's just a staged update/etc perhaps.
<sinzui> rick_h_: I think that was planned for actual searching...bug I have also not committed to searching since that is definitely beyond our need to have the choice secure
<sinzui> thanks rick_h_
<rick_h_> sinzui: sorry, not following on searching/choice secure?
<sinzui> There is a request to let you search for the bug instead of enter a number
<rick_h_> sinzui: ah gotcha.
<sinzui> We are only changing the feature so users do not accidentally link the wrong bug.
<rick_h_> ic, so I can understand the desire for a way to say "search bugs for something like this" and then want to open and make sure it's the bug you're thinking.
<deryck> lifeless, yeah, I've got our call scheduled in about 2 hours.  Still good for you?
<lifeless> yup
<cjohnston> deryck, sinzui, gary_poster, lifeless... Could you guys take a look at https://lists.launchpad.net/launchpad-dev/msg09569.html please and provide some feedback? I'm trying to make the interaction and expectations for the user more clear between Summit and LP.
<deryck> hi cjohnston.  I saw it and marked it to reply.  But was hoping lifeless or flacoste would reply first. :)
<cjohnston> lol
<deryck> cjohnston, quick look at code and I don't see any use for us.  It's just a boolean flag basically, and likely only used by the summit system.
<cjohnston> That's what I suspect, I just want to verify first. :-)
<lifeless> cjohnston: it was added to let the predecessor to summit be told to not accept conflicts with other sessions for that (session,person)
<cjohnston> So that also sounds like there shouldn't be anything that would block me from making text changes.
<deryck> I think you're fine.  It's the sort of special casing we try to avoid these days, but since it's only the summit app using it anyway, I don't see the harm.
<deryck> lifeless, do you feel strongly about it?
<lifeless> about what?
<lifeless> I don't know what cjohnston plans to do :)
<gary_poster> the text change that cjohnston wants to make.
<cjohnston> Just changing the text to better represent how Summit uses it
<gary_poster> "I would like to change
<gary_poster> the wording for the Participation Essential check box to something that
<gary_poster> better explains a "very interested in this topic" instead of saying that
<gary_poster> someone is essential for this topic."
<lifeless> ah
<lifeless> so AIUI that conflicts with how Canonical /uses/ it.
<lifeless> If you're changing how summit uses it, that will need somewhat wider discussion - Launchpad doesn't care, but UDS / ODS / Linaro Connect organisers may.
<deryck> I believe cjohnston is saying Canonical is changing how we use it.
<cjohnston> How does Canonical use it? or does 'Canonical' mean UDS?
<czajkowski> lifeless: cjohnston is developing for linaro/uds  he is one of the developers on summit
<lifeless> czajkowski: I know cjohnston :P
<lifeless> cjohnston: bit of column A, bit of column B.
<lifeless> cjohnston: its used as I said - to guarantee that if person A is so listed for session S, that they won't have any conflict with any other session for which they are also listed as essential.
<czajkowski> lifeless: :)
<cjohnston> lifeless: but only with Summit, nothing else, correct?
<lifeless> cjohnston: thats then used to arrange private meetings, to arrange topic thought leaders for particular sessions etc.
<lifeless> cjohnston: yes, LP does not care about this flag. We would happily delete it entirely.
<cjohnston> I actually am not opposed to that and just having it all inside of Summit ;-)
<flacoste> cjohnston, lifeless: +1
 * cjohnston wants to remove the LP requirement from Summit and just require SSO
<cjohnston> We actually could remove the LP requirement in probably 2 weeks, but there is alot of fight from some people in Canonical for that
<lifeless> cjohnston: what are their concerns?
<cjohnston> lifeless: because blueprints are such an integral part of how things are done, they want to continue to use blueprints to create meetings instead of creating meetings in Summit and then creating a blueprint as needed
<lifeless> why not have summit create the bp for them ?
<cjohnston> is there an API to allow that?
<lifeless> if there isn't there can be
<cjohnston> I *thought* the last time that it was asked there wasn't, but I'm not positive.. but I'd be more than happy to use that if it was created. I don't have time to do the work tho
<cjohnston> I could make it work on Summit tho
<lifeless> cjohnston: it's not a lot of work to do.
<lifeless> cjohnston: but the LP team is equally flat out; I suggest doing it yourself will avoid handofs and queueing, so its much better to do it than not to.
<cjohnston> ya.. one day someone will add a few more hours to the day, then we can all do things to catch up
<lifeless> cjohnston: mmm, what I mean is, if you're estimating 2 weeks to do this thing, and you can find the time to do that, your estimate is low because of the desire to use blueprints being unsatisfied.
<lifeless> cjohnston: you need to allow (say) 2 days to add the API on the LP end to your estimate of the work involved.
<lifeless> cjohnston: if that changes it from 'can do' to 'can't do' - fine. But its not a separate bit of work that can be ignored.
<cjohnston> right... I could remove the LP requirement in 2 weeks... but yes, to make Summit create BPs it would take more time.
<lifeless> flacoste: shall we ?
<deryck> lifeless, hey.  Shall I start a hangout?
<lifeless> sure, or skype - eithe ris good
<deryck> lifeless, use: http://tinyurl.com/orange-standup
<deryck> I invited too
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
<wallyworld_> rick_h_: the new bug picker behaviour is currently "as designed". the bug link is blue (meaning a new page is opened) rather than green and there's also a "new window" icon, so it behaves like the regular picker in that sense if you were to open the details expander and click the blue person link
<lifeless> gary_poster: thanks for the extra detail before, I had not see cjohnston's reply to the thread.
<cjohnston> :-)
<rick_h_droid> wallyworld ok but do we have any other pickers working like that?
<rick_h_droid> if it's not a new inconsistency then cool but it struck me as something I'd not hit befire
<rick_h_droid> bah
<wallyworld_> rick_h_: no other picker only presents a single result. but the person picker has a blue "new window" link in each result
<rick_h_droid> right but to pick a person I click on the results not the submit button right?
<wallyworld_> rick_h_: it serves its purpose for disclosure, which is to allow to user a better chance of not accidentally choosing the wrong bug. any other work will have to be part of bug linking if we get to do that at some stage
<wallyworld_> rick_h_: yes, for a person, you click the person. but here there is only one result so it's not a list as such
<rick_h_droid> ok, well that was part of my asking. to understand the reasoning
<wallyworld_> it's more a "did you really mean this one" type scenario
<wallyworld_> rick_h_droid: i understand the question, it's perfectly reasonable
<rick_h_droid> I just noticed a new 'interaction' unable and the UX side of me went off
<wallyworld_> scope and time constraints prevent it being made "perfect" up front
<rick_h_droid> np cool. thanks for the background
<wallyworld_> rick_h_droid:  that's ok, i also share the concern about inconsistency, but we are unable to address it right now
<rick_h_droid> ok maybe I'll get time to have some you I have fun during our feature work
<rick_h_droid> ui that is
<wallyworld_> maybe :-)
<rick_h_droid> Don't hold my breath you're saying huh
<wallyworld_> one never knows how these things will play out; best laid plans and all that :-)
<wallyworld_> wgrant: any reason distros don't have bug/branch sharing policies?
<StevenK> wallyworld_: http://pastebin.ubuntu.com/1145896/
<wgrant> wallyworld_: They should eventually, but they don't have BVPs/private_bugs at present, so it's not important for the migration or clear exactly how it would work.
<wgrant> The UI will probably be identical.
<wallyworld_> wgrant: cool, just wanted to confirm before i update the ui
<wgrant> Yup
<wallyworld_> StevenK: looks ok
<StevenK> wallyworld_: Yes, reason I'm asking you is the comment and other ff tests
<wallyworld_> StevenK: i don't think the doc tests check the include_description parameter
<wallyworld_> the doc tests probably should be migrated to join the other unit tests at some point
<StevenK> Right, I'm wondering if I care ...
<wallyworld_> always good to delete doc tests :-)
<StevenK> wallyworld_: I can't see ChoiceEdit in lazr-js-widgets.txt
<wallyworld_> i think it's the vocab, let me check
<wallyworld_> StevenK: look for EnumChoiceWidget
#launchpad-dev 2012-08-14
<lifeless> wgrant: Care to do me a favour ?
<lifeless> wgrant: see #isd scrollback and guess what it is.
<wallyworld_> wgrant: i expect we will eventually make bug/branch sharing policy required=True, so for now i should default to PUBLIC if null
<wgrant> wallyworld_: All projects will start with them unconfigured when we turn this on.
<wallyworld_> so unconfigured = PUBLIC, right? i need to determine what to show in the ui if not set
<wgrant> unconfigured = public or private, with a different meaning to any of the existing enum values
<wgrant> unconfigured means that private_bugs and BranchVisibilityPolicy are respected
<wgrant> Which have more complicated meanings than *_sharing_policy
<wallyworld_> oh joy. ok, so that makes things a little more complicated on the ui.
<wgrant> Yeah, indeed, sorry. Forgot about that
<wallyworld_> np, it's all fun
<wgrant> Perhaps makes it similar to the milestone widget
<wgrant> I don't think it needs to be settable back to null
<wgrant> That can always be done through the API if someone makes a mistake and needs to revert.
<StevenK> Perhaps we should add it to the "Configuration options" on the project page
<StevenK> "Configure sharing options"
<wgrant> That's not "Configuration options"
<wgrant> That's "UI trainwreck"
<StevenK> Same same
<wallyworld_> so i was working on adding it to the +sharing page as discussed this morning
<wgrant> (but yes, it's possible that the "(!) Sharing" link should become "(!) Configure sharing" down there, but that's not relevant here)
<wgrant> The sharing config clearly belongs on +sharing
<StevenK> wallyworld_, wgrant: https://code.launchpad.net/~stevenk/launchpad/drop-enhanced_choice_popup/+merge/119461
<wallyworld_> StevenK: the test case needs to be renamed now there are more tests added
<StevenK> wallyworld_: Sorry, was noming. http://pastebin.ubuntu.com/1146190/
<wallyworld_> np
<wallyworld_> that'll do, i already +1
<lifeless> ggrrrr iso-testing.
<lifeless> GRRRRGRGRGR
<StevenK> lifeless: Hm?
<lifeless> bug 6.
<lifeless> Again.
<lifeless> I will bet a large (say 1000) amount of money that there is a doc or some other thing in the iso-testing system that folk are following.
<StevenK> wgrant: Do you want to help me bash my head against this testbrowser upgrade branch, or do you have something more productive for me to do?
<StevenK> With hopefully less pain and suffering.
<wgrant> StevenK: What's still broken?
<StevenK> wgrant: Still the four from yesterday.
<StevenK> xx-person-subscriptions has been nailed shut
<StevenK> Errr
<StevenK> cp -a lib/canonical/launchpad/icing/yui_2.7.0b/build build/js/yui2
<StevenK> cp -a lib/canonical/launchpad/icing/yui_2.7.0b/build build/js/yui2
<StevenK> cp -a lib/canonical/launchpad/icing/yui_2.7.0b/build build/js/yui2
<StevenK> Repeat a lot
<wgrant> StevenK: That's been around for a while.
<lifeless> bigjools: want to do that call from last week ?
<StevenK> wgrant: Bah
<bigjools> lifeless: can do - and I'll explain why I missed it :)
<bigjools> lifeless: call me when ready
<lifeless> bigjools: ok, be a couple of minutes
<StevenK> wgrant: Why are recipes so terrible? Do you think that its plausible that OOPS-71b883b1dbbd619339e84b662eca301a is related to bug 893576?
<lifeless> https://code.launchpad.net/~lifeless/python-oops-timeline/lgpl/+merge/119468
<lifeless> wgrant: did you do that favour for me btw?
<wgrant> StevenK: That OOPS doesn't appear to exist anywhere
<StevenK> Oh, you have to kidding me.
<wgrant> Unless we have an OOPS storm somewhere
<wgrant> lifeless: No
<StevenK> OOPS: OOPS-dabad1c5d11797cefc608105ae54d218
<StevenK> Filename: /srv/oops.canonical.com/oops-amqp/launchpad/production/2012-08-13/OOPS-71b883b1dbbd619339e84b662eca301a
<StevenK> *FACEPALM*
<wgrant> Ah
<wgrant> Yes
<wgrant> And no, they're unrelated
<wgrant> There's a separate bug for that one
<StevenK> Easiest fix is to error out say "Remove the obsolete distro from your recipe." Slightly harder is to filter out the obsolete distro
<StevenK> wgrant: What's your opinion?
<wgrant> Not sure :)
<StevenK> lifeless: ^ You're pretty opinated. :-)
<StevenK> Anything at all is better than an OOPS
<lifeless> StevenK: oops filenames != oops ids
<StevenK> lifeless: Why display them at all?
<lifeless> so that if you want to look at custom fields you can
<wgrant> wallyworld_: Did you try putting the fields in the main body? They would probably fit well, particularly as it doesn't really make sense for them to be in the sidebar.
<wgrant> (and that huge warning won't usually be there)
<wallyworld_> wgrant: it didn't look so good - too much text and vertical space used
<wallyworld_> i hate reading lots of text without it being somehow grouped
<wallyworld_> plus we have wasted white space on the right of the page i thought it may be nice to use, similar to the bug privacy portlet
<wallyworld_> but, all just my opinion
<wgrant> wallyworld_: Yeah, indeed. But I think it could work well to just simply split the page into two sections: the first declares which artifacts are which type, the second declares who can see which type.
<wgrant> It's difficult to say
<wallyworld_> yeah, need user feedback i guess. maybe we can through it out there and see what feedback we get; it's beta after all
<wallyworld_> and the layout can be changed quickly to suit
<wgrant> Indeed
 * wallyworld_ has run out of coffee, situation critical, need to go buy some more
<wgrant> Uhoh
<StevenK> Hm, I thought the feature flags around SPRecipes were long dead
<wgrant> They should be. Why?
<StevenK>     def setUp(self):
<StevenK>         super(TestSourcePackageRecipe, self).setUp()
<StevenK>         self.useContext(feature_flags())
<lifeless> StevenK: can I have a review of https://code.launchpad.net/~lifeless/python-oops-timeline/lgpl/+merge/119468 ?
<lifeless> wgrant: any chance you can review that ppr thing ?
<StevenK> lifeless: I glanced at it. TBH, I thought were going to self-review.
<lifeless> hmm, perhaps I should. But I've interrupted you already, so meg.
<lifeless> StevenK: wgrant: I can't find the bug (but I know it exists) for recipes w/obsolete series.
<StevenK> 890	-# GNU Affero General Public License version 3 (see the file LICENSE).
<StevenK> 891	+# GNU General Public License version 3 (see the file LICENSE).
<StevenK> Tis bug 1006493
<wgrant> lifeless, StevenK: Bug #1006493
<wgrant> Yeah
<wgrant> That
<lifeless> grah. thanks, apparently I fail at simple search and replace.
<StevenK> lifeless: That's buildout.cfg. GPLv3? Really?
<StevenK> lifeless: That's the only thing that belts me in the face. I'd prefer the copyright years updated to include this year, but I'll leave it up to you.
<lifeless> StevenK: no other changes to the file - copyright doesn't accrure.
<StevenK> lifeless: "but I'll leave it up to you." :-)
<lifeless> sure, thanks.
<StevenK> lifeless: And approved.
<lifeless> StevenK: I replied because it would be wrong to update the copyright, as opposed to merely a preference :)
<lifeless> StevenK: thank you!
<wallyworld_> wgrant: do you know if one can call via a web service named_operation a method that has been declared as a property mutator?
<wgrant> wallyworld_: No. You have to use PATCH for that
<wallyworld_> bollocks :-(
<wgrant> Why?
<wallyworld_> patch sucks because it returns an entry resource and not what one might really want
<wallyworld_> whereas a named op you can return arbitrary json data or html or whatever suits your needs
<bigjools> it'd sure be nice if the "workitems_text: Unknown status XXX" was accompanied by a list of valid statuses
<adeuring> good moorning
<lifeless> are there any undeployed DB patches ?
<wgrant> No
<wgrant> cjwatson will hopefully have one soon
<wgrant> It's in review now, I believe.
<wgrant> Which means we can apply the new downtime schedule now :)
<czajkowski> wgrant: has that been blogged/annouced yet ?
<wgrant> stakeholders have been told, and I imagine it will be blogged soonish
<cjwatson> https://code.launchpad.net/~cjwatson/launchpad/db-process-accepted-bugs-job/+merge/119320 awaiting review, yes
<mgz> can feature flags be set in advance of them actually existing?
<mgz> or should I put a note in the NDT for the land that includes this change for the l-osa to set the flag after?
<wgrant> mgz: 'If true, load posts from the launchpad for display on the root page.' â do you mean "Launchpad blog"?
<wgrant> mgz: And yeah, you can set beforehand if you need
<mgz> dammit
<wgrant> Having unknown ones isn't a problem
<mgz> literally just sent that to be merged
<mgz> can I sneak in another rev with ec2 land?
<wgrant> mgz: ec2 land doesn't ask PQM to merge a specific revno. It'll submit whatever's in the branch when the run finishes.
<wgrant> So you can push up a rev now while it's running, as long as you promise it won't break the test suite :)
<mgz> woho! :)
<lifeless> czajkowski: it has now
<czajkowski> lifeless: sweet :)
<czajkowski> lifeless: would you like me to to A) copy that onto the blog and circulate everywhere, or B) just post that link to other places?
<wgrant> If you do copy it elsewhere, might want to s/5 seconds/5 minutes/
<lifeless> czajkowski: Thanks for offering. I'll do an announcement to the main LP blog myself.
<lifeless> wgrant: did I typo ?
<wgrant> "This schedule will give us a max of 30 seconds downtime per day, a
<wgrant> significant reduction from the current maximum of 5 seconds,"
<wgrant> :)
<lifeless> DOHFUCK
<lifeless> wgrant: please follow up to my mail and correct it ;)
<wgrant> Sure
<czajkowski> lifeless: np, I know it's late you side of the world :)
<lifeless> You are as late as you feel.
<mgz> czajkowski: also there's a s/slow/slot/ to fix when reposting
<lifeless> czajkowski: blog post done; feel free to edit if you wish :).
<czajkowski> lifeless: cheers
<G> lifeless: czajkowski: s/at once of these times/at one of these times/ ?
<czajkowski> G: fixed
<wgrant> cjwatson: " * even the API for accepting unapproved uploads to a PPA is cumbersome; while it is (I think) possible, it requires explicitly granting oneself a queue admin permission on one's own PPA"
<wgrant> cjwatson: I don't believe it's possible to create queue admin permissions on a PPA
<wgrant> It's not meant to be, at least.
<cjwatson> wgrant: What's there to prevent it?
<cjwatson> The archive owner has launchpad.Edit on IArchive, and I don't see any other restrictions on newQueueAdmin.
<cjwatson> Not saying it's sane.
<wgrant> :(
<wgrant> Indeed
<cjwatson> Obviously the answer is to not get into that situation in the first place.
<wgrant> Yeah
<czajkowski> cjwatson: you've been busy this morning, :)
<czajkowski> bug queue is busy :)
<cjwatson> czajkowski: only four :)
<mgz> 17477 tests run in 4:12:35.906462, 1 failures, 0 errors
<mgz> dammit... there was a doctest
<mgz> why are you testing html output in doctests launchpad ;_;
<cjwatson> Think of it as a reserve of LoC.
<mgz> ha, a good tip, just found a couple of hundred to lose
<deryck> rick_h_, I'm qa-ok on that edit emails thing.
<rick_h_> deryck: ok cool thanks
<mgz> ...is this google web service stuff actually used any more?
<mgz> ah, it's on the main page only?
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: rick_h | Firefighting: - | Critical bugs: 4.0*10^2
<mgz> actually, I'll be conservative for now...
<mgz> deleting too much junk would make this more annoying to land.
<rick_h_> cjwatson: ping
<cjwatson> rick_h_: yes?  (you have 15 minutes until my screen session goes unattended)
<rick_h_> cjwatson: just wanting to say that I'm looking at your branches for review, but man their a bit out of my wheel house and lots of corner cases it seems
<rick_h_> cjwatson: so I think I'll punt to the more build-side heavy folks, but don't want you to think I'm ignoring you
<cjwatson> that's fine, I'm offline from 15 minutes from now until Thursday anyway
<rick_h_> cjwatson: ok, sorry for the delay
<cjwatson> (I've done my pending QA on dogfood and should be able to qa-ok those changes from my phone once they hit qastaging)
<rick_h_> thanks cjwatson
<czajkowski> cjwatson: off anywhere nice
<czajkowski> or just taking a break from lp :)
<cjwatson> family funeral
<deryck> rick_h_, fantastic work, sir, on the wireframes!
<deryck> rick_h_, can you spare 5 minutes for a video chat about it?  Just next steps, no corrections or comments from me.
<rick_h_> deryck: sure thing
<deryck> rick_h_, standup hangout.... joinging now....
<czajkowski> cjwatson: oh I'm sorry to hear that.
<rick_h_> deryck: should I attach a pdf you think or just refer to the U1 link?
<deryck> rick_h_, refer to the U1 link
<rick_h_> k
<rick_h_> deryck: ok, email sent, I'll be hiding under this desk for the rest of the day :P
<deryck> rick_h_, heh, no fear!
<rick_h_> deryck: woot, and ndt is complete so once I fix these 3.5 bugs I we can flip the feature flag for orange!
<deryck> nice!
<czajkowski> deryck: is https://bugs.launchpad.net/launchpad/+bug/1036267 rolling out soon ?
<deryck> czajkowski, just deployed.
<deryck> rick_h_, don't forget to mark the bugs fix released.
<czajkowski> oh good, before I hop my phone of a wall!
<czajkowski> stupid 2 fctor auth!
<czajkowski> *factor
<rick_h_> deryck: ah, it won't auto do that?
<deryck> rick_h_, no, sorry
<czajkowski> rick_h_: no bugs are lazy that way!
 * rick_h_ has messed that up several times then doh
<rick_h_> thanks for the heads up/reminder then
<cgoldberg> is anyone here familiar with PPR (Launchpad Page Performance Reports)?  It's a zserver tracelog parser that creates html reports (pageperformancereport.py).  lifeless recently moved the code from launchpad codebase into lp-dev-utils... and I'm doing some work on it now.  We want to re-use it for parsing Apache Access logs (for SSO and other projects).
<cgoldberg> anyone up for a code review of a lp-dev-utils branch? :)
<czajkowski> cgoldberg: the on call reviewer is in the topic, currently it is rick_h_ so he'd have to see how busy his queue is
 * deryck goes offline for lunch, back soon
<cgoldberg> czajkowski, thanks
<rick_h_> cgoldberg: linky to the MP?
<rick_h_> sorry, just back from lunch
<cgoldberg> rick_h_,  https://code.launchpad.net/~coreygoldberg/lp-dev-utils/ppr-access-parser/+merge/119409
<rick_h_> cgoldberg: k, loading up
<cgoldberg> rick_h_, k thanks!   lifeless was sorta guiding me.. but got too busy to review an MP.  If he has time today, i'd like him to look also before merging
<rick_h_> cgoldberg: ok, if he's online before I EOD I'll prod him
<lifeless_> science, it works.
<rick_h_> says you :P
<sinzui> rick_h_: are you aware of a yui function/library that we could trivially reuse to set the deactivated attribute of subordinate fields. eg checkbox/radiobutton must be checked for input field to accept input?
<rick_h_> sinzui: not off the top of my head no. I'd htink it's be just an event on a form handling object with a list of disable/enable based on some attrChange event
<sinzui> maybe we want to add an attr to all subordinate fields that cannot have input without the another field's state being checked. eg, data-leader="membership-can-expire" on the field that asks for the expiration days. A common script could look for that attr, from the leader field from add a handler to toggle deactivated based on the state of leader.checked
<rick_h_> sinzui: wallyworld_ jcsackett just a heads up, updating the JS tests to pass in 3.5.1. Some broke since last fix up. https://code.launchpad.net/~rharding/launchpad/yui35_test_fix2/+merge/119603
<rick_h_> so just reminder on the quotes on attrs and such, orange is going to start running with the 3.5.1 feature flag soon and I'll see if I can find a chance to figure out a way to run our JS tests on multiple versions somehow
<rick_h_> sinzui: ah, gotcha. Yea, nothing I know of off the top of my head. Is this on some widget out there?
<sinzui> sorry rick. I reviewed some of those tests you fixed
<rick_h_> sinzui: np, it's going to happen until the tests actual fail for everyone
<sinzui> is the leap to 3.6 harder? should we target that now instead on 3.5?
<rick_h_> sinzui: so two parts
<rick_h_> 1) 3.6. is new and I don't tend to trust new and I don't think we *need* anything in there atm
<rick_h_> 2) I'm working on making it easier to test/upgrade versions and going from 3.5.1 to 3.6 will make a good test run of that process
<sinzui> fab
<rick_h_> sinzui: plus we can't do anything still until we get everyone on the combo loader so it'll be some time yet I'm sure :/
<sinzui> I thought the loader was on for everyone. I thought I was the last person to switch, and I hacked my env to work with combo-loader and ie8 a few weeks ago
<rick_h_> sinzui: if your form needs are a current widget maybe we can do some sort of extension that adds a .submit event binder and runs through a list of 'fields xxx require yyy enabled else set xxx.value(undefined)
<rick_h_> sinzui: well, it's on for up through beta-testers
<rick_h_> but not on full production
<sinzui> okay. understood and agreed we need it on default first
<rick_h_> we're waiting on an RT to setup squid caching of JS combo load files for that
<sinzui> rick_h_: I think I want two data attrs to define two different behaviours. One ensure the subordinate field cannot be used without first checking the master...to prevent users from entering unneeded information. I want the opposite where entering a value in the input checks the master checkbox/radiobutton so that my entry is not ignored by some forms.
<rick_h_> sinzui: yea, I think we'd have to have a base starting widget that was a 'form container'
<sinzui> We have about 5 bugs where form input fails because the form does not do that the user implies, or the form implies the user should do something.
<rick_h_> sounds perfect for something like a yui 3.5 View base class
<rick_h_> *sigh*
<rick_h_> deryck: ok, so since these are just test fixes, going to ask for the feature flag to 3.5.1 to be set on orange. Any last hesitations?
<deryck> rick_h_, none, I laugh in the face of orange squad js risk.  do it!
<rick_h_> lol, ok your the approval person :P
<lifeless_> rick_h_: how would you invalidate the hypothesis that science does not work ?
<rick_h_> you mention 'social science' and laugh and run away? :P
<sinzui> It's all in your head. Like the ending of the Lath of Heaven, we are dying, dreaming of life
<lifeless_> rick_h_: you should read 'thinking, fast and slow'.
<lifeless_> rick_h_: not the social science you think you know.
<rick_h_> lifeless_: sorry, now approaching hour 7 in a car dealership and getting YUI 3.5.1 going into testing 7 months after the sprint to enable the change so I'm frisky today :P
<lifeless> rick_h_: oh la la :P
<rick_h_> deryck: ok, you've got 3.5.1 enjoy
<deryck> I will!
<rick_h_> I'll write up an email to the -dev list saying it's alive and kicking
<sinzui> jcsackett: do you have time to discuss my next actions to make quick team registration sane
<jcsackett> sinzui: sure, just one sec.
<jcsackett> sinzui: i just missed your invite, calling back.
<sinzui> jcsackett: haha. teams have two descriptions and the page hides the screw up. I may need to do some work on Bug #5283 just to keep the LoC count down
<_mup_> Bug #5283: "Home page" vs. "Description" is misleading <bad-commit-11051> <easy> <lp-registry> <qa-ok> <teams> <tech-debt> <ui> <users> <Launchpad itself:In Progress by nigelbabu> < https://launchpad.net/bugs/5283 >
<jcsackett> sinzui: fantastic. :-P
<cgoldberg> lifeless, i'm gonna EOD in a bit.. not sure if rick_h_ got anywhere on my code review :)  could you have a look today?
<cgoldberg> https://code.launchpad.net/~coreygoldberg/lp-dev-utils/ppr-access-parser/+merge/119409
<rick_h_> cgoldberg: sorry, lifeless popped in and he was assigned on the review so passed since it involves amking some config stuff mandatory/etc and I'm not sure where this is used
<rick_h_> so lifeless  ^^
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
<cgoldberg> rick_h_, np
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/format-imports-and-again/+merge/119645 when you have a spare moment. sinzui caused most of the fallout, but oh well
<sinzui> StevenK: you don't need a review of mechanical changes made by a script....particularly when the changes were made by mechanical scripts to begin with
<StevenK> sinzui: Oh yes I do. format-imports has made some bad calls over the six or seven times I've run it over the full tree.
<wgrant> Yeah
<wgrant> I catch an issue about half the time
<sinzui> fix the script
<wgrant> Adding "# FIRST" fixes most files, but not all of them
<wgrant> Anyway, this case is all good.
<StevenK> Yeah, some of the files require manual fixing if they have imports that are between the copyright header and __metaclass__
<wgrant> lint should probably be taught to complain about that
<StevenK> Indeed
<StevenK> wgrant: Thanks, lp-landing
#launchpad-dev 2012-08-15
<lifeless> benji: please don't land that python-pgbouncer branch.
<lifeless> benji: until the question I raise in the review is resolved.
<lifeless> benji: pinging here incase you didn't see the email is all.
<NCommander> WHere did launchpad-buildd move to? Its no longer under lib/canonical/buildd nor can I find it in the source tree
<StevenK> It was split out
<NCommander> lp:launchpad-buildd
<NCommander> Found it, thanks
<StevenK> NCommander: lib/canonical is also emptyish too
 * NCommander notes the wiki pages really could use an update
<StevenK> That was known.
<NCommander> StevenK: so for doing livecd builds on the buildds, I already determined we need a new custom livecd type (raw-livecd). My current idea is to have the CD build infrastructure connected to a devirtualized PPA, and it merely needs to dput a dummy package to spin the livefs's. Does this sound sane? (alternatively, I can have it work in the main archive, but then we have to deal with releasing live images through the queue which will be k
<StevenK> You got cut off
<NCommander> 21:16:02 < NCommander> StevenK: so for doing livecd builds on the buildds, I already determined we need a new custom  livecd type (raw-livecd). My current idea is to have the CD build infrastructure connected to  a devirtualized PPA, and it merely needs to dput a dummy package to spin the livefs's. Does  this sound sane? (alternatively, I can have it work in the main archive, but then we have to
<NCommander>  dall with releasing live images through the queue which will be kinda annoying)
<NCommander> alternatively, I could write a new API to the buildds to build the squashfs's indpendently, but that sounds like a *lot* more pain than its worth.
<StevenK> NCommander: So, I'm not sure.
<lifeless> NCommander: doing a dummy package to trigger builds is fugly and unneeded.
<lifeless> NCommander: where will the [d]debs for the livecd come from ?
<StevenK> ftpmaster.internal, I guess
<NCommander> lifeless: ftpmaster.internal. I already managed to spin a squashfs in a PPA
<lifeless> ok
<lifeless> so, there are already 3 sorts of builds
<NCommander> (I needed a helper package so I could run livecd.sh as sudo, but other than that, no headaches (right up until I hosed the uploader :-))
<lifeless> recipes, translations, packages
<lifeless> add a forth type
<NCommander> recipes?
<lifeless> recipes run on the buildds
<NCommander> is this documented somewhere? :-)
<StevenK> It's probably under buildmaster
<StevenK> lib/lp/buildmaster, that is
<NCommander> Actually most of it is in launchpad-buildd :-/
<StevenK> NCommander: buildd-manager needs to know about the types of build at least
<NCommander> (and at least for TRANSLATIONTEMPLATES its in lp/translations; this code is all over the $#!@# place)
<NCommander> Yeah, ther'es a enum.py which is usually the first place you go so your test cases don't explode with unknown identifer
<StevenK> SPRecipe is probably in lib/lp/code too
<StevenK> I've forgotten its identifier
 * NCommander feels like he's about to be in a lot of pain
<StevenK> Well, duh
<StevenK> Doing CD builds *properly* via LP is a lot of work.
<StevenK> And you have LoC constraints
<NCommander> LoC?
<StevenK> Lines of Code
<NCommander> lines of code?
 * NCommander looks at the wiki
<StevenK> NCommander: New features now have to be LoC neutral
<NCommander> WTF?
<NCommander> what does LoC neutral mean?
<StevenK> NCommander: If you want to add 500 lines of feature code, you have to find 500 lines of something to remove first.
<NCommander> ...
<NCommander> rm -r lp-branches/devel/lib/lp/soyuz/tests :-P
<StevenK> Hah
<StevenK> NCommander: We have a tool that cjwatson wrote. For example:
<StevenK> steven@undermined:~/launchpad/lp-branches/devel% loc-contributions 'Steve Kowalik'
<StevenK> -835
<lifeless> You can get LoC waivers if appropriate
<NCommander> so, let me get this straight. To successfully commit code to Launchpad, I need to step back and delete something else?
<StevenK> Or you get an exception from lifeless.
<NCommander> I rather just modify ubuntu-cdimage to build a deb in a PPA and then wget the deb. a *lot* less pain
<lifeless> or in some other way offset the increased cost of running and maintaining launchpad that the new code brings.
<NCommander> lifeless: adding one machine of every architecture to the buildd pool increasing relaibility and decreasing live image build time?
<NCommander> (the livecd builders can be decommisioned and turned into regular buildds if we don't need dedicated machines)
<lifeless> StevenK: flacoste hands out exceptions too IIRC>
<lifeless> NCommander: as  afraction, how much is the increase there?
<lifeless> NCommander: how does it decrease live image build time ?
<NCommander> lifeless: allows parallization of images across flavors, load balancing, and decreases issues if a buildd machine needs to go offline for service
<NCommander> As it stands, we can build one flavor across all architectures at any given time (ARM has two livebuilders with a rather nasty hack to use both)
<lifeless> whats the utilisation % of the current livebuilders ?
<NCommander> 6-8 hours per day on dailies. Can go up to 100% during freezes if large amounts of respins are required
<NCommander> Spread across 6(?) machines, one per arch one an additional one for armhf
<lifeless> how many machines are there in the current buildd pools ?
<StevenK> For LP, or livebuilds?
<NCommander> two minimium per architecture (I believe we're down a hppa builder, I'll check)
<lifeless> LP
 * NCommander checks the current numbers)
<StevenK> hppa is dead to us
<lifeless> Will Ubuntu want move flavors or more liveimage builds once the latency is reduced ?
<lifeless> StevenK: don't tell lamont
<NCommander> lifeless: that is a discussion the tech board must make. Currently we're allowing new live images on all archs aside from ARM
<NCommander> (the ARM builders barely keep up with the load as is)
<StevenK> 5 i386, 3 powerpc, 3 amd64, 2 ia64, 2 sparc, 1 hppa, 1 lpia and 12 armel, 7 armhf
<StevenK> Hell, lpia is more dead to us than hppa
<NCommander> ARGH, BAD MEMORIES
<StevenK> NCommander: Worse for me
<NCommander> StevenK: well yes but lpia wasn't as bad as psb
<StevenK> Don't use that language with me.
<NCommander> >;-)
<StevenK> Crown Beach is so toally dead to me.
<StevenK> *totally
<NCommander> StevenK: I recommend high explosives personally. Helps get rid of the burn
<lifeless> Ok, so we're looking at 16 hours more build time/day across the board
<lifeless> which is 15% or so on i386, 20% on amd64
<NCommander> lifeless: offset by the fact that every architecture will almsot certianly get an additional buildd (once the livebuild pool is decommissioned)
<lifeless> NCommander: that *is* the additional buildd
<lifeless> NCommander: 24-8 = 16
<NCommander> d'oh
<lifeless> this will make it easier to argue for more scalable capacity too perhaps.
<lifeless> I'm inclined to grant an exception with a caveat: Please *try*, after the project, to reduce LoC impact.
<lifeless> but if you can't, that will be ok.
<NCommander> lifeless: I will, but I'm a heavy commenter
<NCommander> SO as long as its only code I need to elimate, I will try
<lifeless> NCommander: cjwatson is at about -4K lines.
<StevenK> NCommander: refactoring tests is a big win
<lifeless> NCommander: there is a huge amount of FAT in the LP code base.
<NCommander> StevenK: personally, I'd like to add more tests to Soyuz
 * NCommander sucks
<NCommander> *ducks
<NCommander> fuck
<StevenK> Hahaha
<lifeless> NCommander: lots and lots and lots of places that things are redundant, or wasteful.
 * StevenK quotefiles NCommander 
<NCommander> lifeless: personally, I'd love to see LP not use 300+ queries to upload a package, but that's just me
<lifeless> NCommander: see above under fat.
<NCommander> yeah
<NCommander> Thanks for the LoC exception.
<StevenK> NCommander: From our OOPS reports, statement counts: 1350  OOPS-1db714aef8b3b55ac9a7a58cc336ebc4  BugTask:+index
<NCommander> Ow
 * NCommander notes he uses launchpad as proof that postgresql will scale just as well as oracle
<StevenK> 1054  OOPS-861cfa9611c5fbb82f3b97998ea1dcf1  Archive:+copy-packages
<StevenK> But Colin is working on that one
<StevenK> 10000  OOPS-bfc5975ff6ecfe11483e9b28b0f772bf  Unknown
 * StevenK blinks
 * StevenK looks up that OOPS
<lifeless> NCommander: so strictly, we don't insist on before-landing-credit, just that folk keep themselves honest and work on overall shrinking the maintenance burden.
<NCommander> StevenK: why do I feel like that OOPS hit some sorta query limiter
<lifeless> NCommander: what I'm saying to you is I accept the argument that the project is itself a burden reduction, but you should still try to address code fat as you do stuff :)
<StevenK> We don't have a query limiter
<NCommander> lifeless: I may submit mini-branches on the side to delete cruft on the side
<NCommander> That's an oddly round number then
<StevenK> Right
<StevenK> I'm loading the oops
<StevenK> Well, trying to
 * StevenK prods neem
<lifeless> it will take a little
<lifeless> 10K backtraces.
 * NCommander will see StevenK at UDS still waiting :-)
<StevenK> NCommander: I wish
<lifeless> NCommander: for instance:
<lifeless> https://code.launchpad.net/~cjwatson/launchpad/uefi-ppa-no-unapproved/+merge/119542
<StevenK> steven@undermined:~/launchpad/lp-branches/force-ibug-into-line% bzr di -r 14918.. | diffstat -s
<StevenK>  14 files changed, 519 insertions(+), 753 deletions(-)
<lifeless> NCommander: -5 LoC, clearer tests, and bug fix.
<StevenK> That's one of mine. I fixed IBug so it no longer read like a bad romance novel and made the ZCML clearer.
<StevenK> HAhaha
<StevenK> lifeless: Just got a ISE from neem
<lifeless> \o/
<StevenK> "Please contact the server administrator, [no address given]"
<StevenK> Hah, and the error log is useless
<StevenK> Thanks for being so helpful, WSGI
<lifeless> StevenK: it should have oopsed
<lifeless> StevenK: it may still be processing though.
<StevenK> I have no idea, I got a generic ISE page
<StevenK> The error log just says "Premature end of script headers"
<lifeless> yay. Not.
<StevenK> lifeless: So, I guess it didn't OOPS?
<lifeless> I guess not.
<lifeless> the oops occurs when the thing propofates up
<lifeless> no script headers means the app didn't send any before apache flicked it off
<StevenK> wgrant: Can haz review? https://code.launchpad.net/~stevenk/launchpad/no-oops-request-daily-build/+merge/119654
<wgrant> StevenK: Busy now, sorry.
<wgrant> Possibly this afternoon
<StevenK> lifeless: Why more OOPS summaries?
<lifeless> StevenK: Why indeed?
<lifeless> StevenK: I have nil idea.
<lifeless> Ask -ops
<StevenK> wgrant: Are you unbusy now?
<wgrant> StevenK: Not really
<StevenK> :-(
<wgrant> Busy benchmarking paranoia
<StevenK> lifeless: AH, come here
<lifeless> StevenK: ?
<StevenK> lifeless: http://pastebin.ubuntu.com/1148122/
<StevenK> lifeless: Not sure why mailman is there, given lib/mailman.
<StevenK> lifeless: But the main reason I grabbed you is because the last time I tried to rip subvertpy out of sourcecode, things went bang.
<lifeless> trlololol https://plus.google.com/109613336078485786201/posts/1mC8Wm37W2g
<lifeless> oh sorry. right.
<wgrant> StevenK: lib/mailman is built code
<wgrant> StevenK: (yes, our build system writes into lib/)
<lifeless> StevenK: you can't remove mailman like that.
<lifeless> StevenK: :(
<lifeless> StevenK: but testresources should be fine.
<StevenK> lifeless: So I put subvertpy into download-cache, add it into versions.cfg, remove it from sourcedeps.conf, and make, and then nothing can import subvertpy
<lifeless> What steps have you taken t debug this ?
<StevenK> None, because I'm not sure what steps to take.
<lifeless> ok
<lifeless> so when something can't be imported
<lifeless> it may mean its not on the python path
<lifeless> or it may mean that something is broken in the thing
<lifeless> is there an egg?
<StevenK> It's a tarball
<lifeless> is the egg on the path when you look at sys.path under bin/py ?
<lifeless> StevenK: tarball is whats distributed.
<lifeless> eggs are what live in eggs/
<StevenK> eggs/subvertpy-0.8.10-py2.7-linux-x86_64.egg exists
<StevenK> Oh, wait, that's the old one
<StevenK> So, no, no egg
<lifeless> well
<lifeless> tried running bin/buildout ?
<StevenK> make clean && make should do that, no?
<lifeless> have you removed lib/subvertpy ?
<StevenK> I didn't know that was there. Sob.
<lifeless> I note your pastebin is missing the remove of lib/testresources
<StevenK> Right
<StevenK> I'll do that too
<lifeless> (this is why removing sourcedeps is such a great idea :))
<StevenK> I noticed mustache quietly died from sourcedeps
<lifeless> I nuked.
<lifeless> I think.
<lifeless> Therefore I delete.
<wgrant> Well
<wgrant> Um
<wgrant> It was better in sourcedeps than where it is now :)
<StevenK> lifeless: Deleted lib/subvertpy and no dice. Still no egg
<lifeless> no, it was rick that did it.
<lifeless> StevenK: ok, and you ran bin/buildout ?
<lifeless> StevenK: (don't answer by talking about make :))
<lifeless> StevenK: does 'python -c "import subvertpy"' work for you ?
<StevenK> Just ran bin/buildout and still no egg
<lifeless> StevenK: ok, and is subvertpy listed in setup.py ?
<wgrant> StevenK: Did you define it as a dep?
<wgrant> Yaeh
<wgrant> in setup.py
<StevenK> Ah
<StevenK> Let me do that
<lifeless> Now, you may say 'why?' and thats a good question. But until we get the *bzr deps* into setup.py, buildout won't traverse through them to find subvertpy.
<StevenK> lifeless: Right, so indeed, putting it in setup.py was the answer.
<lifeless> cool.
<lifeless> I would add a comment explaining why.
<StevenK> But this branch is at -3/+3 :-(
<adeuring> good morning
<NCommander> How do I get a local LP instance to accept a 3.0 (quilt) source package?
<StevenK> There's a table
<StevenK> NCommander: Have a look for InitializeDistroSeries, it copies stuff into a table that ends with or contains 'Format'
 * NCommander just patched soyuz-sampledata-setupo to give me a precise
<wgrant> Hm, didn't I already fix that?
<NCommander> You go natty->oneiric and forgot precise
<wgrant> Apparently not
<wgrant> Well
<wgrant> I have onerous :)
<wgrant> From before the name was announced
<wgrant> So I haven't updated it in a while.
<wgrant>         ('Onerous Ocelot', SeriesStatus.FUTURE, '11.10'),
 * NCommander remembers how much 'fun' it is to setup LP from scratch
<StevenK> NCommander: Impossible. The brain suppresses bad memories, which is exactly what we are counting on.
 * NCommander misses dak sometimes
<NCommander> s/sometimes/g
<NCommander> seems soyuz dev really wants a SMTP server setup
<NCommander>   File "/usr/lib/python2.7/smtplib.py", line 284, in _get_socket
<NCommander>     return socket.create_connection((port, host), timeout)
<NCommander>   File "/usr/lib/python2.7/socket.py", line 571, in create_connection
<NCommander>     raise err
<NCommander> error: [Errno 111] Connection refused
<StevenK> I get the same issue, ignore it
<NCommander> StevenK: then my GPG key doesn't import
<StevenK> Are you using utilities/make-lp-user?
<NCommander> documentation for soyuz sampledata says it should add my key to ppa-user
<NCommander> It imports my SSH key just fine
<NCommander> there
 * NCommander does a test build
 * NCommander discovers Launchpad rejects uploads to security if the archive is in DEVELOPMENT, and goes to modify the changelog to try again
 * NCommander groans
<NCommander> deb http://archive.launchpad.dev/ubuntu precise main
<NCommander> What do I need to fix that? :-/
<NCommander> nm found the magic UI page with the value I had to change
 * NCommander has the basic framework in launchpad-buildd to build live images, and is just making sure he didn't break the world when I wasn't looking
<StevenK> NCommander: I do wonder how you're going to implement for-project
<NCommander> StevenK: flavor argument passed into the xmlrpc for launchpad-buildd
 * NCommander already has that done and working
<StevenK> I meant inside LP
<NCommander> StevenK: new API in all likelyhood controlled by a celeberity
<StevenK> Ew, celebrity
<NCommander> StevenK: I wanted to use dummy packages, much easier :-P
<StevenK> NCommander: And then Colin will drive to your house and stab you.
<NCommander> But basic design idea I had with ubuntu-cdimage uses launchpadlib to tell LP to kick a build for a given architecture/flavor/series/etc, get the librarian URL for the image file when done, wget it, continue as normal
 * NCommander *loves* we have three different livefs image builders
<NCommander> live-build/livecd.sh/ubuntu-defaults-image
<NCommander> here's what the arguments look like for a LiveImage build: http://paste.ubuntu.com/1148377/
<NCommander> StevenK: if celeberities are bad, what should I key squashfs's builds off of? We don't want people willy-nilly being able to smack to respin API
 * NCommander could connect it to the distro driver (~ubuntu-drives)
<NCommander> *drivers
<StevenK> Probably ~ubuntu-cdimage in the short term
<NCommander> I was personally thinking ubuntu-release
<NCommander> or a new team, since I though hardcoding ubuntu stuff in LP was a bad thing
<NCommander> though then again, ubuntu itself is a Celebrity ...
<lifeless> wgrant: busy?
<wgrant> lifeless: Hi
<lifeless> hi
<wgrant> lifeless: You pinged?
<lifeless> I did
<lifeless> cgoldbergs thing; I'm interviewing folk for 3 of the 4 hours I'm working tomorrow - half day
<lifeless> I'd really like it to get a paranoid once over.
<wgrant> Ah. I might look tomorrow, if you're busy
<lifeless> I'm running in place.
<lifeless> (yes, am busy, or wouldn't be asking)
<wgrant> k
<NCommander> wgrant: StevenK what dispatches things like TranslationTemplate builds?
<lifeless> builddmaster
 * NCommander has a fairly decent grasp on how jobs go from LP->slaves now, but isn't quite sure what triggers the jobs (aka, how we geting */model/*buildbehavior.py)
<lifeless> NCommander: not sure why you talk about a celebrity above.
<lifeless> NCommander: any permissions you need should be distro centric, to support derived distros (e.g. linaro).
<NCommander> lifeless: building a livefs will require building on devirtualized builders, and not connected to any package
<lifeless> and?
<NCommander> lifeless: what team should it be connected to. There's isn't exactly a nice place where to hang it (first though was ~ubuntu-drivers but that team is already overloaded as is)
<NCommander> Unless I add a new Image Builder or Release Team thingy somewhere in a distroseries
<lifeless> any published distro driver would be my starting point; creating distros is a privileged operation.
<lifeless> don't overthink this.
<NCommander> lifeless: right, but the driver team currently has 60+ members in it and is being used for fun things like UDS managers
<lifeless> you don't need to limit it to 'just the people that have to do it', what you need to do is to 'exclude folk that might abuse or attack it'
<lifeless> these are wildly different problems.
 * NCommander is about halfway getting launchpad-buildd to understand livebuilds, and suspect I'll need help when we get to the plumbing side in LP
<lifeless> If a UDS manager spins an extraneous livecd build, who cares. Tell them off. Problem solved.
<NCommander> Good point
<NCommander> lifeless: thanks for your help
<StevenK> lifeless is just bored because he can't play CS:GO.
 * NCommander personally recommends civ 5 :-)
<StevenK> I was playing that a few nights ago. Bloody hell, it's *enormous*
<NCommander> That game is a soul-sucker.
<NCommander> Works amazingly well under wine though. Was pretty suprised when the demo just went
<lifeless> StevenK: helping and being bored are not mutually exclusive ;)
 * NCommander notes his end goal is to have a nice webpage that completely replaces nusaken with all image building tasks being done on LP
<NCommander> vs. the arcane voodoo required now
<NCommander> *nusakan
<NCommander> so flavors can click a button and presto; respun images
 * NCommander grumble
<NCommander> *s
<NCommander> my builder went to an ABORT state and I can't figure out where the log file is
<wgrant> NCommander: There's no build log for an ABORTed builder
<wgrant> NCommander: Check /var/log/launchpad-buildd/default.log on the slasve
<NCommander> wgrant: thanks. So my build succeeded and uploaded but running the test cases for launchpad-buildd sent it straight into OMG broken mode
<jam> mgz: did your patch make it to prod?
<jam> mgz: ah, looks like it is in 15805 waiting for the next NDT
<NCommander> exceptions.ValueError: Slave is not BUILDING when told build is complete - *grumble*
<wgrant> NCommander: Ah, yeah, running the test suite on a machine with a live launchpad-buildd is a good way to die.
 * NCommander doesn't really feel like setting up a second machine at the moment to run as a slave :-/
<wgrant> NCommander: VMs :)
<rick_h_> deryck: ping for standup
<deryck> rick_h_, coming now, sorry
<rick_h_> anyone have a link handy for using the people.canonical space?
<rick_h_> I think I just sent a test .png file to the middle of no where
<czajkowski> rick_h_: jpds might know
<rick_h_> thanks czajkowski
<czajkowski> np
<deryck> yay for stable wifi at the wife's shop finally.
<rick_h_> yay
<czajkowski> sinzui: what happens when you click sub someone else https://answers.launchpad.net/launchpad/+question/205885
<sinzui> czajkowski: js failed to init on the page I think
<mgz> <https://answers.launchpad.net/launchpad/+question/205885/+addsubscriber> is a 404 (js-less) alright
<czajkowski> sinzui: odd it's happening on https://answers.launchpad.net/launchpad/+question/205854 also
<sinzui> czajkowski: because is a js error
<czajkowski> nods
<czajkowski> just randomly happened
<sinzui> js fails every where, not on one page
<mgz> well, the page not existing isn't a js error... if there's no non-js mode, there shouldn't be a href pointing to lies
<sinzui> czajkowski: The subscribe someone else is shared with bugs and mp reviews
<sinzui> since js is broken, we can see that answers does not support that via a html view
<sinzui> The error in the page related to comments. maybe jcsackett can understand the error on questions
 * sinzui adds a comment to a question to see if the js starts to work
<jcsackett> sinzui: with a comment it should.
<sinzui> yep
<rick_h_> yea, there's no .boardComment and the JS doesn't allow for that to not exist so it's dying
<jcsackett> looks like a goof in the cl setup--it assumes (and our test provides) a boardcomment.
<rick_h_>             var container = Y.one('.boardComment').get('parentNode');
<jcsackett> sinzui: i can whomp up a fix for that today.
<rick_h_> assume it must exist
<sinzui> czajkowski: subscribe js on answers is broken when there is not comments
<sinzui> I added a comment, reloaded the page and js worked
<czajkowski> oh most odd
<czajkowski> thanks
<sinzui> I will report the bug. The js just needs to return early if there is no work to do
<jcsackett> sinzui, rick_h_: looks like the cl can just not exist if it doesn't find boardcomment, since there's nothing for it to do in that situation anyway. agree/disagree?
<rick_h_> jcsackett: yea assuming there's no way to ajax add comments
<sinzui> jcsackett: this is true for answers, Mp and bugs start with comments I think
<rick_h_> jcsackett: that would blow up if a CL wasn't there for it to get added to
<jcsackett> rick_h_: there's no interaction like that to worry about; there is an ajax add comment form, but when you add a comment on the page you can't immediately hide it anyway. that may be a defect, but it's a separate one as it's long been the case.
<sinzui> jcsackett: This has been broken for weeks. I think subscribing before commenting is rare
<jcsackett> sinzui: i concur. and re: bug's having comments off the bat, that's correct, but i don't believe they are shown as boardComment--the first "comment" is the description.
<sinzui> ah, correct
<sinzui> jcsackett: you can report the bug and optionally choose to fix it. "disclosure information-type regression"
<jcsackett> sinzui: ok, i can create a fix for this pretty easily.
<sinzui> jcsackett: since there re no reviewers today, I pre-emptively ask you to review a branch. I am not asking you to review it now
<jcsackett> sinzui: no problem.
<jcsackett> sinzui: r=me.
<sinzui> thank you jcsackett
<jam> lifeless: you might be interested in: https://code.launchpad.net/~jameinel/launchpad/loggerhead-clear-cache/+merge/119793
<jam> calling '.clear_cache()' on all the versioned file objects has a pretty big impact on the 'spider loggerhead' peak memory.
<lifeless> does this impact performance? was the cache getting used before ?
<jam> lifeless: every request gets a new bzr_branchd
<jam> so no
<lifeless> cool then
<jam> (we could try to re-use them, but that is much hairier than not :)
<lifeless> see you guys on the flip side.
<NCommander> so how do files leave librarian?
 * NCommander is trying to figure out how old squashfs's will poof out of librarian
<StevenK> NCommander: They are garbage collected when they are not referenced.
<NCommander> StevenK: what defines a reference? (or is there a minimium age before they get GC'ed?)
<NCommander> Mostly I want squashfs's to stick around for a day or two then I don't care if they get deleted
<StevenK> NCommander: So, stuff in the the librarian is an LFA and a correspending LFC (LibraryFileAlias, and LibraryFileContent)
<StevenK> NCommander: Things like BugAttachment link to LFA.
<NCommander> Sounds like squashfs's need a LFC when they're uploaded that points to the most recent squashfs
<StevenK> If the attachment is deleted, the row in BugAttachment is dropped, and the LFA now has nothing pointing to it
<StevenK> And it will get GC'd after a stay of execution, which is a week or something
<StevenK> NCommander: Your custom upload will create the LFA and LFC. Don't worry about that.
<NCommander> eh, in that case, its mostly then I don't care. I just need to be able to find the latest squashfs in librarian, download it, and let it get executed whenever its time rolls around
<NCommander> StevenK: er? I though non-package uploads just go straight into librarian and bypass the queue. I can't have a custom upload without an assiocated source package, no?
<NCommander> (aka, how translations are handled currently)
<StevenK> NCommander: Just because they're directly ACCEPTED does not mean there isn't a PackageUpload for them.
<NCommander> so launchpadlib call to trigger a build -> live-build job runs on buildds -> live-image upload to Soyuz -> PackageUpload?
 * NCommander was thinking the system worked trigger->buildd job->librarian
 * NCommander does have launchpad-buildd at the point where it now can do a live-image build
<StevenK> I
<StevenK> Sigh
<StevenK> I'm trying to remember if the translations stuff has a PackageUploadCustom associated with it
<NCommander> raw transitations do, which then via some automagic process get queued up by buildd manager to be executed. THere seems to be *very* little code in soyuz about it
<NCommander> The processed translations, I'm not so sure
<StevenK> NCommander: The revelant code would be in buildmaster
<NCommander> there's also very little code there :-/
 * NCommander has heard of trying to find a needle in a needlestack before ...
<StevenK> Perhaps archiveuploader
 * NCommander thinks he needs to run a translations job and watch to see where the files go
<NCommander> If they go into soyuz, its easier because then I can simply iterate on an archive for the proper custom upload type (per docstories)
<StevenK> NCommander: I think you need to write a LEP
<NCommander> There's a partial one for this usecase
<NCommander> https://dev.launchpad.net/Soyuz/Specs/BuilddGeneralisation
<bigjools> you want a new build type?
<NCommander> I implemented a new build-type. live-image
<huwshimi> rick_h_: Just found some internet, if you're available for a call sometime. Otherwise I'll send you an email.
<NCommander> WHich partially works
<NCommander> I think its stuck in an infinite loop ATM though
<bigjools> it's not an easy task
<NCommander> bigjools: well, my build-type works, the trick is getting ways to trigger the builds via launchpadlib, and getting those files somewhere useful
 * NCommander originally was just going to have a source package + custom upload 
<StevenK> wgrant: http://pastebin.ubuntu.com/1149798/
<wgrant> NCommander: Why are you trying to use a packageupload?
<wgrant> It's not a packageupload
<wgrant> And I'm not sure there's any reason for it to be.
<NCommander> wgrant: I didn't think it was, StevenK was the one who brought it up
<wgrant> :(
#launchpad-dev 2012-08-16
 * NCommander feels like this entire system is pretty much an undocumented blackbox
<wgrant> It's undocumented, but not really a blackbox :)
<NCommander> I'm trying to figure out where the files returned from gatherResults() go
<wgrant> NCommander: That's on the slave, right?
<NCommander> Yeah
<NCommander> sourcepackagerecipe is easy, it generates a _sources.changes and uploads it normally. I'm at a loss on how translation-templates does its magic
 * NCommander has kinda figured out where in LP the translations build magic is, but is still trying to grok is
<NCommander> *it
<NCommander> currently my code managed to install livecd-rootfs, build the image, bombs on gatherResults() and goes to chrootFail() :-)
<rick_h_> huwshimi: yea, give me a sec, just got to the coffeeshop
<huwshimi> rick_h_: It's ok if this is a bad time. I can even annotate your mockups if that's easier.
<rick_h_> huwshimi: https://plus.google.com/hangouts/_/5b5b820d6987551f9545ee30c79ed37cd5c3e30a?authuser=0&hl=en-US#
<StevenK> wgrant: Do I need to rip out part of IBug.transitionToInformationType()?
<wgrant> StevenK: For the bug supervisor stuff? Not yet.
<StevenK> wgrant: So it's just what is listed in the bug? That's +1/-45, and from what I can see, only 1 failed test.
<wgrant> StevenK: Did you also remove the helper methods on the notificationrecipientset>?
<StevenK> wgrant: Yah, and dropped it from the ZCML
<wgrant> Great.
<StevenK> Just waiting on test -vvm bugs
<StevenK> Oh, now to find the code that gives the bug supervisor a structural subscription
<NCommander> Ok, good, my code now executes all the way through without crashing anything
<NCommander> Now to figure out how t get files to go somewhere
<StevenK> wgrant: Right, I guess the self.addBugSubscription(bug_supervisor, user) can die too
<wgrant> StevenK: That's a second branch, but yeah
<wgrant> StevenK: Once that dies, the limitation on setting the value can die
<StevenK> In the second branch?
<wgrant> Since it will just convey privilege, not INFINITE SPAMMAGE
<wgrant> Right.
<StevenK> Right, I'll not worry about that bit yet.
<StevenK> wgrant: .... No failed tests
<wgrant> StevenK: Heh
 * NCommander finally found a translations buildjob (https://launchpad.net/~vcs-imports/nautilus/master-git/+translation-templates-build/24952), but still can't tell what it uploaded
<wgrant> NCommander: The job's gatherResults implementation calls slave.addWaitingFile, which causes it to be returned when buildd-manager asks what files are waiting.
<NCommander> Right, that part I got
 * NCommander is trying to work out the interface for BuildFarmJob, and where in LP said jobs are scheduled; its not in buildmaster according to grep
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/no-more-add-bug-supervisor/+merge/119818
<wgrant> NCommander: The buildfarm infrastructure requires three classes from you on the master side: an IBuildFarmJob (eg. BinaryPackageBuild), an IBuildFarmJob (eg. BuildPackageJob -- as the name suggests, it's deprecated but still required), and an IBuildFarmJobBehaviour (eg. BinaryPackageBuildBehaviour)
<NCommander> right
<NCommander> I can see that
 * NCommander is *slowly* beginning to wrap his head around the monster
<NCommander> Its actually not that bad once you know where all the pieces are I suppose
<StevenK> wallyworld_: Since wgrant is such a bad person, can you review https://code.launchpad.net/~stevenk/launchpad/no-more-add-bug-supervisor/+merge/119818 ? Should be incredibly difficult.
<NCommander> so I need to implement the correct classes (I'm guessing in soyuz/model unless you can think of a better place), and then pin out to launchpadlib so I can trigger said builds
<NCommander> wgrant: am I roughly on the right gametrack?
<wgrant> NCommander: Yup
<wallyworld_> StevenK: ok
<NCommander> wgrant: seems relatively straightforward. What's the catch? :-)
<wgrant> NCommander: You have to touch our code :)
<StevenK> Hahah
<NCommander> as long as I don't need to deal with the UI bits, I'll live
<NCommander> Soyuz isn't as bad as other code bases I dealt with
<NCommander> (i.e. dak (before I destructively rewrote it a large chunk of it)
 * NCommander is wondering how hard adding a new launchpadlib API will be ...
<wgrant> Not very
<wallyworld_> StevenK: done
<StevenK> Depends where you want to staple it in
<wgrant> Not just less code -- less email :)
<NCommander> email?
<wgrant> Talking about StevenK's branch
<StevenK> NCommander: My branch, not your stuff
<NCommander> oh
 * NCommander wonders if someone has already ripped out the old bounty code
<StevenK> NCommander: https://code.launchpad.net/~stevenk/launchpad/no-more-add-bug-supervisor/+merge/119818 if you want to celebrate
<StevenK> The bounty tables are long dead, wgrant and I killed them last year
<NCommander> StevenK: I would, except LP just timed out on me :-/
<NCommander> Or my browser did, not sure which
<StevenK> NCommander: Diff against target: 98 lines (+2/-48) 3 files modified
<NCommander> Net -46 LoC, woo
 * NCommander can't even remember if the old bounty system was ever actually used
<StevenK> I now have 4 branches in ec2.
<StevenK> +54, +9, +1, and now -46.
<StevenK> I get to rip a little more out
<NCommander> what's the handy-dandy bzr command to see how many LoC lines have changed?
<StevenK> bzr di | diffstat -s
<StevenK> Or bzr damage, which is an addon
<NCommander> mcasadevall@perdition:~/src/launchpad-buildd$ bzr di | diffstat -s 3 files changed, 168 insertions(+)
<NCommander> And that's just for a bare minimium implementation
 * NCommander groans
<StevenK> That doesn't really count, it's in -buildd
<NCommander> oh
<NCommander> Thought it was still considered for part of the LoC cost to LP
 * NCommander decides its now a good time to take a break from banging my head on a wall
<StevenK> wgrant: I can't figure out where the code that restricts setting bug supervisor is
<wgrant> StevenK: It's possible that I was misremembering, and the problem was that there was no restriction.
<wgrant> Ahhh
<wgrant> It's in the view
<wgrant> BugRoleMixin.validateBugSupervisor
<StevenK> I thought it was in the view, but ProductEditView is sort of ... empty
<StevenK> Right, another +3/-61
<NCommander> when was the neutral LoC policy put in place anyway?
 * NCommander returns
<wgrant> wallyworld_: Heh, bug #1037364
<wgrant> I thought there'd be some stupid-long titles :)
<wallyworld_> wgrant: examples work for me, but i have a wide monitor. sigh 2012 and people still use narrow screen resolutions
<wallyworld_> i can make the number of lines before truncation 3 or 4
<wallyworld_> just for bug titles
<StevenK> Sigh, ec2 needs a timeout for connecting to an instance
<wgrant> StevenK: It does
<StevenK> wgrant: I guess most or all of TestBugSupervisorEditView can die
<wgrant> StevenK: I haven't looked, but probably.
<wgrant> StevenK: It becomes a thoroughly thoroughly boring edit view.
<StevenK> wgrant: So we need to test it works, but the oddball tests of "You can't set that team" just die.
<wgrant> Right
<StevenK> wgrant: less-sourcedeps == 17481 tests run in 4:12:31.684115, 0 failures, 0 errors
<wgrant> StevenK: Nice.
<StevenK> wgrant: So, how do I run the bzr-svn tests?
<wgrant> StevenK: Not sure.
<StevenK> Oh my god, can I delete lib/lp/bugs/tests/has-bug-supervisor.txt for being hideous.
<wgrant> Yeah
<wgrant> The only thing that still looks relevant is the ForbiddenAttribute
<wgrant> Well, and I guess setBugSupervisor might not be tested elsewhere
<StevenK> setBugSupervisor just sets the attribute, what the heck is to test?
<StevenK> wgrant: self.assertRaises(
<StevenK> Sigh
<StevenK>         self.assertRaises(
<StevenK>             ForbiddenAttribute, setattr(self.target.bugsupervisor, None))
<StevenK> That doesn't work, but ForbiddenAttribute is thrown
<wgrant> StevenK: self.assertRaises(ForbiddenAttribute, setattr, self.target.bugsupervisor, None)
<StevenK> wgrant: http://pastebin.ubuntu.com/1150046/
<StevenK> wgrant: It run the tests for the mixin :-(
<wgrant> StevenK: Stop the mixin from inheriting TestCase
<StevenK> wgrant: Right, I think I have mostly everything passing, but ec2 can pick that out.
<wallyworld_> wgrant: does this look ok for adding InfoType embargoed? https://pastebin.canonical.com/72290/ it needs a mechanism to decide which projects are allowed to be configured to use it, not sure what the rules should be for that
<wgrant> wallyworld_: The proposed mechanism is that we just don't show Embargoed sharing_policies in the UI unless they're selected
<wgrant> The PES setup script will set it through the API
<wallyworld_> wgrant: ok, so i just remove the entry from POLICY_ALLOWED_TYPES and it's ok then?
<wgrant> wallyworld_: No, you need that.
<wgrant> And you need POLICY_DEFAULT_TYPE for that too
<wgrant> Those two dicts define what the model will allow branches to be if that setting is selected
<wgrant> Not which branch_sharing_policies are shown in the UI
<wallyworld_> what about POLICY_REQUIRED_GRANTS
<wgrant> Default should be Embargoed, required should probably be Proprietary, I think
<wgrant> It's possible required should be Proprietary OR Embargoed, but that's easy to change later if we need to
<wallyworld_> that's what i was wondering
<wallyworld_> i'll start with just proprietary
<wgrant> Yeah
<wallyworld_> and i fixed getInformationTypesToShow so that it doesn't show up in the ui, removed embargoed from shown_types
<wgrant> wallyworld_: I think Embargoed should be in shown_types
<wgrant> Whenever it's available
<wgrant> Just like Proprietary
<wallyworld_> yes, but that is done near the botton of the method
<wgrant> The only difference between the two should be that one of the sharing_policies isn't shown in the UI
<wallyworld_> the current product policy is added in
<wgrant> That doesn't sound like getInformationTypesToShow
<wgrant> You mean the thing to get the sharing policies to show?
<wallyworld_> it's in BranchEditFormView
<wallyworld_> yes
<wgrant> Right
<wallyworld_>     def getInformationTypesToShow(self):
<wallyworld_>         """Get the information types to display on the edit form
<wgrant> That sounds sensible
<wgrant> What's the change you've made?
<wallyworld_> cool, i'll add some tests and do a mp, thanks
<wgrant> 14:31:19 < wallyworld_> and i fixed getInformationTypesToShow so that it doesn't show up in the ui, removed embargoed from shown_types
<wgrant> That's the wrong way around
<wgrant> It should be in shown_types, just like Proprietary is
<wgrant> shown_types defines what the options are in the UI, assuming that the branch is able to have those types
<wgrant> if a project has Embargoed enabled, Embargoed should be shown on Branch:+edit
<wallyworld_> ah, getting myself confused jumping between irc and code. i know what i need to do
<StevenK> Switch to vim?
<wallyworld_> how would that help anything?
<StevenK> It surely would. Both wgrant and I can agree on that.
<wgrant> Definitely.
<wgrant> Vim solves world hunger
<wgrant> Vim even stops the boats.
<wallyworld_> a sensible refugee policy would stop the boats
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/no-structsub-for-bug-supervisor/+merge/119831
<wgrant> wallyworld_: Now you're getting it. Therefore vim is a sensible refugee policy.
<wgrant> StevenK: +55? What is this?
<StevenK> wallyworld_: Well, they don't travel to the US, because they know that all the US is going to do is ship them right back. We just detain them for a while.
<wallyworld_> maybe if we forced refugees to use it they might not want to come over
<wgrant> StevenK: What sort of notifications do other similar edit forms show?
<wallyworld_> we detain them and give them big ass tvs and mobile phones and health benefits
<wgrant> StevenK: I wonder if we can drop these ones.
<StevenK> Just drop the notifications entirely?
<StevenK> Trying to think what we have like it.
<wgrant> StevenK: Also, since it's probably not exposed, you might be able to remove setBugSupervisor entirely
<StevenK> It is
<wgrant> Since the new implementation is pretty boring
<wgrant> Is it exposed as setBugSupervisor?
<wgrant> it's probably a mutator
<StevenK> @mutator_for(bug_supervisor)
<wgrant> Yeah
<wgrant> So it can be removed.
<StevenK> Should I do that in this branch?
<wgrant> Might as well
<StevenK> Lots of places call setBugSupervisor
<wgrant> Will mitigate the tests you added.
<wgrant> (by rendering them redundant)
<wgrant> (and failing)
<StevenK> Both of them, in fact
<wgrant> (and removable)
<StevenK> 15 files changed, 17 insertions(+), 359 deletions(-)
<StevenK> Now to delete every setBugSupervisor call ever
<wgrant> :)
<StevenK> wgrant: Hmmm, how does the ZCML change?
<StevenK> setBugSupervisor is listed seperately
<wgrant> StevenK: setBugSupervisor is probably in the launchpad.Edit attributes= section. Replace it with bug_supvisor in set_attributes
<wgrant> Obviously spelt with enough letters
<StevenK> wgrant: Like so? http://pastebin.ubuntu.com/1150072/
<wgrant> StevenK: That's the one.
<StevenK> Er, what
<StevenK>     def _create_scenario(self, user):
<StevenK>         with person_logged_in(user):
<StevenK>             logged_in_user = getUtility(ILaunchBag).user
<StevenK> ...
<StevenK>                 user=logged_in_user)
<StevenK> wgrant: Is it just me, or that just pointless and user=user would be fine?
<wgrant> StevenK: LaunchBag.user may not always be the logged in user.
<StevenK> Twitch
 * StevenK leaves it alone for being magical
<wgrant> It should be the same, but I suspect it isn't always
<StevenK>  39 files changed, 84 insertions(+), 477 deletions(-)
<StevenK> wallyworld_: :-(
<wallyworld_> huh?
<StevenK> wallyworld_: I was seeing if we could go a full day with only me landing branches and you wrecked it.
<wallyworld_> oh, i thought i had broken the build or something
<wallyworld_> you just made me load buildbot
<StevenK> Hahahaha
<wallyworld_> bastard!
<wallyworld_> StevenK: my branch in bb now is a diff of one character :-P
<StevenK> Heh
<wallyworld_> my other ones had a merge conflict in ec2 i didn't notice :-(
<StevenK> Come on, diff!
<StevenK> Damn it
<StevenK> wgrant: If you're done watching *redacted* for the moment, that diff is updated.
<wgrant> StevenK: Thanks, looking
<wgrant> wallyworld_: I think you're overstating your work a bit.
<wgrant> wallyworld_: It was a single *bit* of diff :)
<wallyworld_> yes, so it was :-D lol
<StevenK> Really?
<StevenK> 3-6 is not one bit
<wallyworld_> 2 became 6
<StevenK> 010 goes to 110
<StevenK> Oh, so it is
<StevenK> I thought it was 3, bleh.
<wgrant> StevenK: Did you look at the notifications?
<StevenK> wgrant: I did not -- what do you think?
<wgrant> I forget what other forms do
<wgrant> I suspect that a lot of them don't notify.
<StevenK> wgrant: Kill it or leave it for sinzui?
<wgrant> We can always remove it later :)
<wgrant> Sorry, looking at your MP again now
<wgrant> Distractions abound
<StevenK> It's you, so duh. :-)
<wgrant> wallyworld_: Hm, actually, Embargoed's addition will also need a small DB patch (since we hardcode (3, 4, 5) in a couple of places), though it can be applied live whenever so it isn't blocked by the current fdt embargo
<wallyworld_> wgrant: i can land the branch first i think
<wgrant> Yeah
<wgrant> Embargoed branches just won't be visible to anyone but admins in searches
<wgrant> And if any exist on production we'll need to poke them with a stick after the patch to make them work.
<StevenK> wgrant: No review?
<wgrant> StevenK: pgkillactive ate my homework
<StevenK> Sure sure
<wallyworld_> in that case  i might do the db patch first
<wgrant> wallyworld_: No need. Embargoed can't be set unless the sharing_policy allows it, and only commercial admins can set sharing_policy
<wgrant> So unless One of Usâ¢ does bad things, Embargoed can't exist on prod yet
<wallyworld_> ok. it should be a very simple db patch, so hopefully can be done soon
<wgrant> Yeah
<wgrant> Just need to find things that hardcode (3, 4, 5) and replace them with (3, 4, 5, 6)
<StevenK> wallyworld_: Change the comments too
<wgrant> I suspect we'll eventually keep a private flag around to eliminate that, but we don't have such a flag atm.
<wallyworld_> in the db patch, sure?
<wgrant> BAH
<wgrant> Firefox
<wgrant> stop crashing
<wgrant> Oh
<wgrant> I reenabled Firebug
<wgrant> That's right
<StevenK> Haha
<StevenK> wgrant: Still no review?
<wgrant> StevenK: r=me, but you should be able to delete a bit more
<wgrant> Heh
<StevenK> Heh, there's another +7/-41
<wgrant> StevenK: What'd you do?
<wgrant> makeProduct?
<StevenK> And dropped the two tests you suggested.
<StevenK> wgrant: http://pastebin.ubuntu.com/1150152/
<wgrant> StevenK: Great.
<StevenK> wgrant: Oh, is there a bug for this bug supervisor mess?
<wgrant> StevenK: Bug #483521, bug #595933, bug #688956, bug #855121, bug #1029724, bug #113825 are possibly relevant
<wgrant> You've fixed at least some of them
<StevenK> wgrant: Indeed, let me link some of them
<adeuring> good morning
<cjwatson> oww, landing Launchpad branches in an environment that disallows outgoing ssh is an unpleasant experience
 * cjwatson tries to cobble together something with paraproxy before giving up and sorting out an all-out VPN
<czajkowski> gary_poster: what is a stretch goal, longer than a week ?
<gary_poster> czajkowski, officially, it means that we don't expect to get it done this week, but it's possible we could get to it if we do particularly well--if we stretch ourselves.  unofficially, it means that I set the goal before I realized how little staff availability we had this week, and adjusted the goals after the fact as I was writing the blog post. ;-)
<czajkowski> gary_poster: ahh :)
<czajkowski> sounds good to me
<gary_poster> :-)
<czajkowski> gary_poster: shall me telling mrevell this in future :)
<gary_poster> czajkowski lol
<mrevell> :)
<deryck> Morning, all.
<czajkowski> deryck: ello
<rick_h_> morning
<cjwatson> stub,lifeless: it's wound up being unclear in +activereviews due to benji's null code approval, but https://code.launchpad.net/~cjwatson/launchpad/db-process-accepted-bugs-job/+merge/119320 is pending a db review
<benji> cjwatson: sorry if I messed that up a little by doing that
<cjwatson> I don't know either way - just occurred to me that the DB reviewers might be skimming over it
<stub> Nah, I just have a crap filing system.
<wgrant> 3
<wgrant> bah
<stub> I guess we want the id column purely because we share code with existing job stuff
<cjwatson> That may actually have been inertia/cargo-culting on my part
<cjwatson> But AFAICS all other job tables have id
<cjwatson> Do you want me to try removing it?
<deryck> adeuring, rick_h_ -- will have to be offline for 25 minutes here shortly, to catch ride with Wendy into the shop.
<adeuring> deryck: ack
<rick_h_> deryck: rgr
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: jcsackett | Firefighting: - | Critical bugs: 4.0*10^2
<cjwatson> NCommander: FWIW I don't think it will make sense to put all of cdimage in LP even in the long term.  I think the right long-term approach is a web service to control image build scheduling (perhaps offspring) that uses LP for auth and for livefs builds.  The rest of it wouldn't gain much in the way of benefit from being in LP; we'll be more agile outside it.
<cjwatson> NCommander: (Also, you may not be aware that we've had quite a few discussions about this recently in UE; and furthermore that I'm in the process of rewriting cdimage in Python for related-to-this and other reasons.)
<cjwatson> stub: OK.  Does http://bazaar.launchpad.net/~cjwatson/launchpad/db-process-accepted-bugs-job/revision/11832 look OK on top of your previous review?  My SQL is way weaker than it should be, but this passes tests.
<stub> cjwatson: That looks fine to me. You ran all the tests?
<stub> cjwatson: I'm happy for you to land either version.
<cjwatson> I ran the tests in my code branch that'll go with that DB branch
<cjwatson> Well, the ones that were added/changed
<cjwatson> But nothing else should be referencing that new table so that should be good enough
<stub> Yeah, I don't think we have any code discovering job tables and doing stuff with them.
<cjwatson> ... I hope
<cjwatson> If there is something insane, EC2'll catch it anyway
<mgz> you hope :)
<jml> is the 'merge queue' stuff on branches ever used by anyone?
<mgz> I don't think so.
<jml> in launchpadlib, is there a way to get the underlying dict of the object?
<jml> x._wadl_resource.representation seems to do the trick
<sinzui> jml: that is exactly what I do in my python that feeds my js scripts
<jml> yeah
<cgoldberg> jcsackett, got any time for a review ?   I have an MP to lp-dev-utils
<cgoldberg> I have an update to lp-dev-utils that allows page-performance-reports.py (PRR) to parse Apache Access logs.  I'm trying to get it deployed this week for some account services apps.
<jml> sinzui: I was just sitting here thinking, wouldn't it be great if launchpadlib were just about dicts and functions
<sinzui> +1
<jcsackett> cgoldberg: sure. link?
<jcsackett> cgoldberg: nm, found the MP.
<cgoldberg> jcsackett, thanks.  https://code.launchpad.net/~coreygoldberg/lp-dev-utils/ppr-access-parser/+merge/119409
<cgoldberg> jcsackett, back in a few mins if you wanna discuss anything
<jcsackett> cgoldberg: ping.
<cgoldberg> jcsackett, pong
<jcsackett> cgoldberg: so, if i'm understanding this correct, you're adding this functionality to get performance reports from entirely different services? it seems like it might be cleaner to just create a new script.
<cgoldberg> jcsackett, my thoughts at first also.. but lifeless thought it should just be integrated with a new [parser] option... 90% of the script is the aggregation and stats logic..
<jcsackett> cgoldberg: fair enough. in that case, i'm writing up some notes i'll post to your MP.
<cgoldberg> jcsackett, great thanks!
<jcsackett> cgoldberg: comments on your MP. needs a bit of fixing, but nothing major.
<cgoldberg> jcsackett, k.. will look and fix up
<cgoldberg> jcsackett, regarding MP comments.  the create_request_class function was done for exactly what you described
<cgoldberg> the servers this will run on will not have launchpad  or dependent code installed.. so importing from zc isn't really possible
<jcsackett> cgoldberg: that seems like another really strong reason to not use lp-dev-utils.
<cgoldberg> jcsackett, i don't disagree :)  I'll talk to lifeless
<jcsackett> cgoldberg: i don't want to block you longer on this; if lifeless has already indicated this should be part of lp-dev-utils, i would say just fix the other import issue i highlighted and leave an XXX comment on create_request_class indicating it needs to exist for the zc import reasons but could be cleaned up if we separated these functions out, ok? (XXX comment style can be found here: https://dev.launchpad.net/PolicyandProcess/XXXPolicy)
<cgoldberg> jcsackett, thanks much.. that sounds good.
<cgoldberg> i'll add comment and re-push
<jcsackett> cgoldberg: yw. ping me when you've done that and i'll mark as approved. also: you can skip the bug part, since there's not a clear bug to my mind, it's just a non-ideal setup.
<jcsackett> (the bug=nnn bit in the comment)
<cgoldberg> gotcha
<jml> I am a horrible person: http://paste.ubuntu.com/1151164/
<jml> sinzui: ^^
<sinzui> jml: I think that style is easier for most developers to adopt.
<jml> sinzui: if I did it right, the number of HTTP requests would be deadly obvious, or at least easily discoverable.
<sinzui> jml: the sharing service we wrote ignore the model. It focuses on your intent and lets you perform bulk actions on collections. Our js just pulls dicts from urls
<jml> sinzui: "sharing service"?
<sinzui> The API service we wrote to share information types with people. Visit the +sharing page of any of you projects. The page doesn't have data in it. The js pulls what it needs to get just what it needs quickly
<jml> sinzui: oh cool.
<jml> sinzui: I was secretly hoping it was actually a separate service
<jml> I guess lifeless would say that my API should probably also specify which scalar values I want
<jml> from previous conversations about DB querying
<sinzui> Maybe. We wrote our methods to return the dicts we knew were needed. The service is not arbitrary. Your does not need to be arbitrary, and maybe it should not expand everything given API my change and your script will slow down
<deryck> jcsackett, hi there.  I have a branch for review if you have time.
<lifeless> jml: mail me about your API, if you want an eyeball cast on it
<jml> lifeless: ok.
<lifeless> flacoste: http://bmfiddle.com/
<sinzui> That must be an American company. "Fiddle Now!"
<sinzui> jcsackett: do you have time to review https://code.launchpad.net/~sinzui/launchpad/change_branch-info-type/+merge/120011
<jcsackett> sinzui: yes.
<jcsackett> deryck: btw, missed your earlier ping, but your branch is r=me.
<jcsackett> sinzui: looking at yours now.
<deryck> jcsackett, thanks!
<jcsackett> sinzui: r=me.
<sinzui> thank you jcsackett
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
<NCommander> cjwatson: I was aware of the efforts to rewrite cdimage in python, and I knew that having LP doing livefs builds were discussed (as far back as a few years ago).
<NCommander> cjwatson: generally when it comes to web-based infrastructure I like to have as few different tools as possible. Launchpad is responsible for packages (and soon livefs building), I rather not split it out into another tool that has to existed in a void. Launchpad is the proper place for it, if anything offspring should be merged into LP for an intergrated experience
#launchpad-dev 2012-08-17
<StevenK> wgrant: I thought bug supervisor turned up on IProduct:+edit ?
<StevenK> Ah
<StevenK> Configure bug tracker
<StevenK> IE, UI trainwreck
<cjwatson> NCommander: but the trend in Launchpad itself is to start trying to split things out into microservices where possible, rather than embedding them all in a single giant webapp
<cjwatson> (and for those to be affiliated to Launchpad in the necessary ways)
<NCommander> cjwatson: er, wasn't one of the motivations of writing launchpad in the first place to prevent having a million microapps (i.e., bugzilla+dak)
<NCommander> I don't personally care one way or another, but having centralized one-stop place for packages/images/builds/archives makes a lot of sense to me
<lifeless> what did I miss ?
<wgrant> 13:10:00 -!- lifeless [~robertc@125.7.18.100] has quit [Ping timeout: 252 seconds]
<wgrant> 13:11:24 < cjwatson> NCommander: but the trend in Launchpad itself is to start trying to split things out into microservices where possible, rather than embedding them all in a single giant webapp
<wgrant> 13:12:18 < cjwatson> (and for those to be affiliated to Launchpad in the necessary ways)
<wgrant> 13:23:44 -!- lifeless [~robertc@125.7.18.100] has joined #launchpad-dev
<lifeless> wgrant: thanks
<lifeless> NCommander: separate codebase != separate UI || siloed behaviour
<lifeless> NCommander: LP set out to create an integrated set of behaviours
<NCommander> Which it succeeded at
<lifeless> NCommander: separate services provide natural scaling and HA boundaries, assuming the services are well defined
<lifeless> NCommander: bugzilla vs dak is not the same as microservices making up the whole product.
<NCommander> Fair enough
<lifeless> NCommander: for an example of another microservice approach, look at s3 + ec2 + cloudscaling etc
<NCommander> THat being said, I stilla rgue that ISO mastering would fall under Launchpad; image builds are already dependent on the LP service and this provides ways for flavors to control their own respins/etc.
<lifeless> consistent feel, integrated management consoles and so on, but very focused facilities that are orthogonal
<lifeless> I think iso mastering should move onto the regular buildds.
<cjwatson> No.
<cjwatson> The live filesystem component should move onto the regular buildds.
<cjwatson> It doesn't make sense for the whole of ISO mastering to do so.
<lifeless> cjwatson: whats the difference there, and why?
<cjwatson> The line I'm drawing is the part that can be run architecture-independently, and that benefits from being on a machine with a full local mirror etc.
<cjwatson> The live filesystem component is the part that must be run on a system of the target architecture.
<cjwatson> And that certainly belongs on the buildds.
<lifeless> cjwatson: you need to run the arch indep part one per arch though, right ?
<cjwatson> Yes, but there are useful economies of scale.
<lifeless> cjwatson: from bandwidth access to the mirror ?
<cjwatson> For example.
<cjwatson> More like same filesystem, since some ISOs use hardlink trees.
<lifeless> cjwatson: ah, interesting. I haven't climbed into that code.
<lifeless> cjwatson: thanks.
<cjwatson> Maybe the importance of this will change as more things move to live CDs, but I don't want that to entirely dominate the design.
<lifeless> cjwatson: do you reuse the hardlink trees for things on different days ?
<cjwatson> I'm not sure I understand the question.
<lifeless> IIUC you build N > 1 ISOs using hardlink trees for common content  ?
<cjwatson> We reuse the mirror.  But hardlink trees are cheap to build so we just do that afresh for each image build.
<lifeless> When you build the same targets the next day, do you build fresh hardlink trees, or evolve the prior one ?
<cjwatson> The former, then.
<lifeless> anyhow. I'm actually arguing for horizontal scalability and automated from-scratch provisioning of the build system, rather than 'use the buildds' per se.
<lifeless> closer to stateless etc etc etc
<cjwatson> In terms of scaling, building live filesystems massively dominates; we're a long way off the point where the central system has any trouble keeping up with the extra bits and pieces of ISO mastering it does at the rate that separate builders can deliver live filesystems.
<cjwatson> Obviously that changes at some point horizontally but it's quite some distance away.
<lifeless> kk
<NCommander> I was more considering the possibility of removing squashfs builds are a seperate step and merely master an entire ISO in a single go on a buildd
<NCommander> */2 cents*
<cjwatson> I know you were, but I don't think that's desirable at this point.
<lifeless> cjwatson: what downside do you see?
<cjwatson> See above.
<cjwatson> In any case I don't think this should even need to be considered for livefs-in-LP work.
<cjwatson> As far as that's concerned, the build job should be calling BuildLiveCD or some equivalent.
<lifeless> cjwatson: above you discussed efficiencies, but unless you hit a bottleneck that doesn't translate to a specific or even necessary downside.
<cjwatson> What it outputs is the business of the job and the consumer, not of LP.
<cjwatson> lifeless: The downside is that we should be moving to livefs-in-LP without having to refactor the entire cdimage build system along the way.
<cjwatson> And it's actually really quite a considerable amount of code to move around.
<lifeless> cjwatson: so, two separate [potential] migrations, rather than one that moves + rearranges? If so I concur.
 * NCommander agrees with that, as moving the squashfs building is considerably less diserpative ATM
<NCommander> and as I said in the beginning, long-term [potential] goal
<lifeless> cjwatson: also, wth are you awake?
<cjwatson> *You're* asking me that? :-)
 * NCommander is dealing with a rather mysterious issue that the buildd complains UNKNOWNSUM
<NCommander> WOrks with a precise chroot sha1sum, breaks with a lucid one
<cjwatson> (Random insomnia)
<NCommander> grep "No URL" -r *
<NCommander> er
<NCommander> oh
<NCommander> the only reason my trigger code worked was the precise chroot was cached in the filecache for the buildd
<NCommander> cjwatson: do we have any images that still require livecd.sh? (I have a handler that calls that, live-build, and ubuntu-defaults-image)
<StevenK> lifeless: I see your iwl is as stable as ever.
<lifeless> StevenK: I'm on ethernet
<lifeless> StevenK: Hotel ethernet.
<StevenK> That's as stable as iwl
<StevenK> wgrant: What sort of cruft is sinzui talking about it bug 1036189?
<wgrant> StevenK: eg. if I set my project to only use Proprietary bugs and branches, and I change all the Private and Private Security artifacts to Proprietary, there's no need to keep Private and Private Security around
<wgrant> The APs and APGs can be deleted.
<StevenK> That sounds more like a job Job rather than a garbo
<wgrant> StevenK: We'd have to trigger it on every transitionToInformationType
<wgrant> So no
<StevenK> Right
<StevenK> wgrant: So if it's a garbo what do I search on?
<wgrant> StevenK: Hm?
<StevenK> wgrant: So if I want to pick products that need AP{,G}s deleted, how do I do it?
<wgrant> StevenK: Find APs that are forbidden, and then find the subset that are unused
<StevenK> wgrant: Right, so find APs that do not make sense given what the product is set up for.
<wgrant> StevenK: Right
<lifeless> wgrant: StevenK: and you won't let folk file bugs that noone-can-see?
<wgrant> lifeless: We will, but the project owner would have to explicitly revoke the permission
<wgrant> The default is always that the owner can see each type
<lifeless> note too the race conditions at the time of revoking
<wgrant> (and the owner gets a warning on +sharing if they do that)
<wgrant> Oh?
<lifeless> new bug of the type the permission is being revoked for
<lifeless> owner may think there are none, but \/ races
<wgrant> We don't prevent filing a bug that nobody can see
<lifeless> sure
<wgrant> If the project owner wants to do stupid things, we let them do stupid things, but they can easily recover
<lifeless> saying that doesn't mean humans won't assume ...
<lifeless> same race exists for setting the project to only proprietary, if you do stuff inline, of course.
<wgrant> Changing the allowed types (eg. setting the sharing policy to Proprietary only) doesn't affect existing artifacts.
<wgrant> But it does stop you from making new ones of the illegal types
<lifeless> cool
 * StevenK tries to cook a query
<StevenK> wgrant: So I'm looking for projects where bug_sharing_policy or branch_sharing_policy isn't PUBLIC and they have APs for PRIVATESECURITY and USERDATA. I'm guessing it's a little more complicated, but it seems those are the only policies that are excluded.
<StevenK> wgrant: Right?
<wgrant> StevenK: If you look near the top of lib/lp/code/model/branchnamespace.py you'll need the dict of allowed types
<wgrant> s/need/see/
<StevenK> Does that also follow through for bugs?
<wgrant> There's a similar one for bugs in a branch that I haven't landed yet. It's basically identical.
<wgrant> wallyworld_: In the sharing_policy = PUBLIC garbo branch, would it be better to check that the project is set to public already (ie. not private_bugs and not exists branchvisibilitypolicy) rather than checking for the lack of a commercial subscription? If we're landing this now, it'll also mean that commercial project setup changes now.
<wgrant> I'd suggest we defer landing it until new projects are being set up using sharing_policy
<wallyworld_> wgrant: i'm not sure why commercial project setup would change? the job only updates non commercial projects
<wallyworld_> we discussed the need to do this in the call - we wanted to ensure that the many projects were migrated for people before beta
<wgrant> wallyworld_: If a project happens to accidentally be created non-commercial and only made commercial later, the old commercial setup procedure will appear to work, but do nothing
<wgrant> private_bugs will be set, but bugs will be public
<wallyworld_> sure, but if a project is made commercial, it would need to have it's sharing policy set at the same time
<wgrant> Right, so we'd need to advise the people who do commercial setup (probably just czajkowski and us nowadays?) to unset sharing policies
<wgrant> That may be acceptable, but we need to say that first :)
<wallyworld_> yes, curtis is all over it
<wgrant> Sounds good, then. But we do need to check that the project is actually configured to be public, not just that it has no commercial subscription.
<wgrant> Or we may be in for a nasty surprise if anything's misconfigured.
<wgrant> Since commercial admins can set private_bugs and BVPs without a commercial sub
<wgrant> So who knows what madness lies out there today...
<wallyworld_> by "public" you mean check the licence?
<czajkowski> wgrant: morning
<spm> heya czajkowski
<wgrant> wallyworld_: Needs to have private_bugs = false, and there must be no BranchVisibilityPolicies at all
<wgrant> czajkowski: Hi
<spm> gentles all; is it still possible for people and/or teams to have name aliases? I can't seem to find any such UI modification for same any more? possibly it was only for projects??
<wallyworld_> wgrant: so theoretically only commercial projects can have private_bugs = true right?
<wgrant> spm: People/teams never could
<wgrant> spm: Projects can
<czajkowski> I believe myself and mrevell will be talking to people who have commercial projects next weeek and going over the new setup with them once myself and mrevell know more and know fully about the set up
<spm> wgrant: right. ta. I misrecall. thanks mon
<wgrant> wallyworld_: Sort of
<wgrant> wallyworld_: "theoretically" in that "in an ideal world, there's no reason for it to be any other way"
<wgrant> wallyworld_: But it's by no means enforced or even suggested anywhere in LP that that should be the case
<wgrant> As a commercial admin, I can go over to Random Open Source Project and set private_bugs and make all branches private only to canonical.
<wgrant> In the old model
<wallyworld_> and that would be a mistae, no?
<wgrant> By policy, yes.
<wgrant> But it will work absolutely fine.
<wgrant> And LP won't warn you that you've done something that doesn't make sense.
<wgrant> And isn't meant to be supported.
<wallyworld_> so if it's not supported then the garbo job can do what makes sense, or?
<wgrant> wallyworld_: It's not supported, but it may well exist and if it does then the current garbo algorithm will cause private stuff to become public.
<wallyworld_> "unsupported" private stuff
<wgrant> s/it may well exist/there is no reason it wouldn't exist/
<wgrant> Sure
<wgrant> But someone making a silent misconfiguration when touching privacy is not a valid reason to make all their stuff public quietly :)
<wallyworld_> ie if people don't have a commercial subscription, can they reasonably expect their stuff to be private?
<wgrant> wallyworld_: If they set private_bugs to true and created private BVPs and LP didn't complain and it worked for 5 years, then yes.
<wallyworld_> how would they set private bugs to true? i thought only an admin could do that?
<wgrant> wallyworld_: We have admins all over the company
<wgrant> (commercial admins, not just full admins)
<wgrant> OEM, HWE, and even non-Canonical groups like Linaro have commercial admins
<wgrant> LP doesn't make the supposed rules clear, so assuming that they've been followed is... not exactly reliable.
<wallyworld_> hmmm, ok. i initially saw the issue as "freeloaders" no longer getting something they hadn't paid for
<wgrant> Well, there's that side of the issue.
<wgrant> But that's relatively minor
<wgrant> compared to the risk of making confidential information public without telling anyone
<wallyworld_> i can do a followup branch then to tighten the rules for what projects are migrated. custis seemed happy with the current approach when we discussed it during the call
<wgrant> launchpad_dogfood=# SELECT COUNT(*) FROM product WHERE private_bugs AND NOT EXISTS (SELECT 1 FROM commercialsubscription WHERE commercialsubscription.product = product.id);
<wgrant>  count
<wgrant> -------
<wgrant>      8
<wgrant> (1 row)
<wgrant> That branch is dangerous :)
<wallyworld_> fsvo
<wgrant> They're all canonical/linaro projects
<wgrant> So by definition not freeloaders
<wallyworld_> do i really need to check bvp or is private_bugs sufficient?
<StevenK> wgrant: I've been staring at this query for like ten minutes and still doesn't make sense. I know what I'm trying to say, but not how to say it. :-(
<czajkowski> welcome to my world StevenK :)
<wgrant> StevenK, wallyworld_: gimme a sec
<wgrant> wallyworld_: There are 30 projects with private, private only, or forbidden BVPs and no commercial subscription
<StevenK> czajkowski: Haha
<wallyworld_> wgrant: ok, i'll add in bvp checks also
<czajkowski> bvp?
<wgrant> wallyworld_: Thanks.
<wgrant> wallyworld_: I didn't think there'd be this many, tbh :/
<wallyworld_> branch visibility policy
<czajkowski> ahh yes *headdesk*
<wallyworld_> czajkowski: they're going away soon so don't even bother to try and understand them :-)
<wallyworld_> wgrant: i'd be ok to ignore bvp = public?
<wgrant> wallyworld_: Yeah, projects that have become public afterwards can't have their BVPs removed, so they're just all set to public
<wgrant> wallyworld_: A project behaves as if it has no BVPs if it only has public BVPs
<wallyworld_> ok, so bvp > 1 in my query
<wgrant> A single Private, Private Only or Forbidden rule means it's not public
<wallyworld_> yep
<czajkowski> have folks come acorss liciencing choices/discussions  before on LP https://bugs.launchpad.net/launchpad/+bug/1037685
<StevenK> wgrant: So no help for me? :-(
<wgrant> StevenK: Sorry, what do you have so far?
<StevenK> wgrant: Roughly 'store.find(Product).find(Product, ' because I've trying to figure out how the next bit should look. :-(
<StevenK> czajkowski: So project licensing we don't really tend to care about -- we really only care is it an open source license or not?
<wgrant> StevenK: So, one thing you could do is batch through the products, grab their APs, compare them to the set
<czajkowski> StevenK: cant really reply with I don't care now can I :0
<czajkowski> :)
<wgrant> StevenK: It's only going to be garbo-daily, so it doesn't need to be a stunningly fast single query
<StevenK> wgrant: Right, I figured that bit
<wgrant> StevenK: Then you compare the information types from the access policies with the ones allowed by *_sharing_policy. Any that don't match the sharing_policy, you query accesspolicyartifact to see if there are any artifacts using it. If not, you delete the APG and AP
<wgrant> s/APG/APGs/
<StevenK> wgrant: Yes, I understand that bit, what I've been stuck on is how to tackle the query that finds if there is work to do.
<wgrant> I'd just iterate through all the products
<wgrant> There's not very many in the scheme of things.
<wgrant> And we only need to do this dailyish
<cjwatson> NCommander: no, livecd.sh is historical interest only.
<NCommander> cjwatson: well, we still use it for old images, and generally, I think we want the capability to respin those until they leave support, no?
<adeuring> good morning
<cjwatson> NCommander: so the way it works is that we always build live images in a chroot of the relevant series, with the relevant series' livecd-rootfs installed
<cjwatson> NCommander: the only piece that's series-independent is BuildLiveCD
<cjwatson> NCommander: if at all possible I'd strongly recommend that you simply call BuildLiveCD
<NCommander> I'm aware. I'm re-implementing BuildLiveCD into the buildd slave.
<cjwatson> Is there a reason to do that rather than calling it?
<cjwatson> It has quite a bit of distro-specific logic.
<NCommander> BuildLIveCD can't be series-independent because the chroots are always presisene unles syou want to pull it in from a PPA or other location
<cjwatson> BuildLiveCD isn't stored in the chroot.
<NCommander> In addition, there isn't a lot of distro-specific logic, and by having the interface one level up, its easier to pass new options to build images or other engineering fun
<NCommander> cjwatson: so then we want to install it on every buildd by hand? (or make it part of launchpad-buildd?
<cjwatson> It's installed on every buildd by hand at the moment.
<cjwatson> I guess I can see the logic in reimplementing it.  If you do, you need to keep all the things it does, including, yes, livecd.sh.
<cjwatson> Damn, now that bizarre failure I was seeing in EC2 has infected buildbot.
<wgrant> cjwatson: Which?
<wgrant> I don't see one
<cjwatson> lp.soyuz.browser.tests.test_archive_webservice.TestArchiveWebservice.test_getAllPermissions_constant_query_count
<cjwatson> on db-devel
<wgrant> cjwatson: It's odd that it's doing both cookie-based sessions and OAuth
<wgrant> But what's the extra query?
<cjwatson> Maybe; but the test is that there are the same number of queries for one perm vs. two.
<cjwatson> Trying to reproduce locally at the moment.
<wgrant> I tend to just decrease the limit in the test to get it to give me a dump
<cjwatson> Yeah.
<wgrant> Then compare to the buildbot failure, and fail to work out what has changed :)
<cjwatson> I'm mostly waiting for my beard to grow while make runs, at the moment.
<wgrant> Heh
<wgrant> make compile is pretty fast
<wgrant> and usually sufficient for non-launchpadlib tests
<cjwatson> This is a launchpadlib test.
<wgrant> launchpadlib, or the test suite's webservice client thingy?
<cjwatson> LaunchpadWebServiceCaller plus launchpadlib_for, apparently.  I was slotting into the existing TestArchiveWebservice.
<wgrant> Ah
<cjwatson> But I specifically wanted to ensure there was no late evaluation hidden in lazr.restful.
<wgrant> WebServiceCaller doesn't need WADL, launchpadlib_for does
<wgrant> Right
<cjwatson> It must be at least slightly random, because my second attempt to get uefi-ppa-no-unapproved through EC2 succeeded.
<wgrant> cjwatson: Right, this sort of thing often depends on whether you're the first webservice test in the process or not
<wgrant> Oh
<wgrant> This is probably complaining there's one too few queries.
 * wgrant counts..
<wgrant> No, one too many :(
<wgrant> There goes the session secret theory
<cjwatson> Mm, it's expected != other.
<cjwatson> Oh.  If I put another list(self.main_archive.getAllPermissions()) at the start of the test, that triggers the failure.  So that suggests it's something a bit like what you're suggesting.
<cjwatson> Also, is lp.buildmaster.tests.test_builder.TestSlave.test_status_after_build a known-flaky test?
<cjwatson> The extra query in the second run is "134-2193@SQL-main-master SELECT Component.id, Component.name FROM Component WHERE Component.id = 1 LIMIT 1".  Oddly, it's listed at the start.  I wonder if it's a leftover from the first run somehow
<cjwatson> Also odd because there's already a "134-138@SQL-main-master SELECT Component.id, Component.name FROM Component WHERE Component.id = 1 LIMIT 1" in the first run.
 * cjwatson discovers record_two_runs, whose purpose seems to be to deal with this.
<jml> we'd like to get https://code.launchpad.net/~james-w/udd/binary-scanning-series/+merge/119248 landed, but I've forgotten how to do it for udd (tarmac? pqm? merge & hope?)
<maxb> Plain old bzr
<cjwatson> wgrant: https://code.launchpad.net/~cjwatson/launchpad/testfix-getallpermissions/+merge/120094
<cjwatson> (or any other reviewer really, but I was talking to wgrant about this above)
<wgrant> cjwatson: Looking
<wgrant> Well, once the diff is there
<wgrant> Ah, it won't be there for a while
<wgrant> fdt
 * wgrant looks at loggerhead
<wgrant> cjwatson: r=me. I didn't know about that nice helper.
<cjwatson> nor did I before grep happened to turn it up :)
<bac> hi adeuring are you reviewing today?
* bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: bac | Firefighting: - | Critical bugs: 4.0*10^2
<adeuring> bac:sorry, forgot to change the toipic..
* adeuring changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: bac, adeuring | Firefighting: - | Critical bugs: 4.0*10^2
<bac> adeuring, quite a few branches on +activereviews.  hope we can knock those out today.
<adeuring> bac: ok, i'll start with "fix-.pocket-queue-admin-series"
 * deryck switches work locations, back online shortly
<cjwatson> adeuring: Thanks for your review of fix-pocket-queue-admin-series.  While I don't object to making the change you mentioned for clarity's sake, you said that it was in LP's style guide, and I can't find it anywhere in https://dev.launchpad.net/PythonStyleGuide or in at least the first few linked pages I looked through.  Could you point me to what I missed?
<adeuring> cjwatson: let me see (actually, I claimed that this rule exists, but I haven't looked at our style guide for longer time...)
<adeuring> cjwatson: https://dev.launchpad.net/CodeReviewChecklist?highlight=%28elif%29
<cjwatson> aha, thanks
<adeuring> though that might be cruft...
<adeuring> cjwatson: the idea idea of the rule is that it should be clear that all possible cases of the if/elif/elif... are covered -- just prevent "casual" errors
<cjwatson> aye
<cjwatson> I've put it back now
<sinzui> jcsackett: do you have time to discuss the IBranchEditableAttributes interface. Project maintainers don't have permission to use change the attrs. This is a problem since the branch is shared with the project, and they cannot respond to cases where data is disclosed, or the branch owner is no longer associated with the project.
<sinzui> jcsackett: I want you to read the interface before we talk because I was surprised by some of the attrs in it
<jcsackett> sinzui: i'll put up the file now, and ping you in a few.
<jcsackett> sinzui: short read. g+?
<sinzui> yes
<jcsackett> sending invite momentarily.
<sinzui> bac: do you have time to review https://code.launchpad.net/~sinzui/launchpad/project-branch-permissions/+merge/120219
<cjwatson> Is there some way to get my testfix from devel earlier merged over to db-devel so that its buildbot can start working again?
<cjwatson> Actually, isn't buildbot-poll supposed to do that?
<wgrant> cjwatson: buildbot-poll will do it if the db-devel builder isn't in proper testfix mode
<lifeless> benji: added
<benji> lifeless: thanks
<cjwatson> wgrant: I fear it may be in testfix mode, though
<cjwatson> So does it need somebody to merge manually?
<wgrant> cjwatson: Not sure if codehosting's still up, but indeed.
<wgrant> You can either merge the testfix branch, or just say the whole stable merge is a testfix.
<cjwatson> wgrant: with bzr lp-land or something more arcane?
<wgrant> cjwatson: I'd just pqm-submit stable directly
<wgrant> bzr pqm-submit -m '[testfix][r=foo] Merge db-stable rWHATEVER' --public-location=bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable --submit-branch=bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-devel
<wgrant> or so
<cjwatson> ok, will have a look in a bit, modulo whatever bits of LP are still alive
<cjwatson> wgrant: not being a reviewer, can I use r=wgrant?
<wgrant> cjwatson: Be my guest
<cjwatson> done
<wgrant> Thanks.
 * cjwatson slightly freaked out by glimpsing "bzr: ERROR: no such option: -0" in PQM output
<wgrant> I thought I'd removed that check.
<wgrant> Maybe not
<wgrant> It's pointless and doesn't work any more, anyway
<NCommander> cjwatson: ok, now that I slept on it. WHy do we want to keep BuildLiveCD? The API I have for Launchpad lets the client choosing the build choose the chroot and the series its building (by default, whatever series you choose to build will be built in said chroot aka; lucid build in lucid using livecd.sh, precise build in precise using live-buid (or ubuntu-default-image)
<NCommander> Is there another piece of functionality I'm overlooking?
<cjwatson> It may not be too outrageous to replace BuildLiveCD with something equivalent, indeed.
<cjwatson> Now that I've thought about it, our process for modifying that involves RT.  Changing that to involve landing an LP branch might actually be an improvement.
<cjwatson> Would be easier to see what's currently running.
#launchpad-dev 2012-08-19
<lifeless> is there a good server credentials wallet service around?
<wgrant> lifeless: What do you mean?
<lifeless> something like keyring but language agnostic, safe on multi-user environments, simple API
<wgrant> There's a reason launchpadlib used python-keyring
<lifeless> I know we have something in-house, not sure if its homebrewed or not.
<wgrant> There's nothing good
#launchpad-dev 2013-08-12
 * pinky is away: Away
#launchpad-dev 2013-08-13
<stub> So yeah, buildout is failing without network. Yay.
<stub> What is weird is I can seem to build the individual eggs in isolation
<stub> So if I build .egg files myself, I can stick them in lp-sourcedeps and things work without a network connection
#launchpad-dev 2013-08-14
<wgrant> jelmer: MP was coming, was just sorting out the bug #1072461 as well.
<_mup_> Bug #1072461: Git HTTP imports don't fetch new revisions <code-import> <git> <Launchpad itself:Triaged> <https://launchpad.net/bugs/1072461>
<wgrant> :)
<StevenK> Dear StormStatementRecorder, why are you allergic to ws_object?
<jelmer> wgrant: :-) Nice find!
<anastaji> Hello, anybody to talk with?
#launchpad-dev 2013-08-15
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/distroseries-spec-preload/+merge/180046
<wgrant> StevenK: What about searches that don't need workitems?
<wgrant> And also I think your preloading omits a sort.
<StevenK> wgrant: I should order by workitem.sequence ?
<StevenK> wgrant: I was thinking about that -- which calls don't you think require workitems?
<wgrant> StevenK: The web UI's search, for example.
<wgrant> StevenK: You should probably aggregate the workitems by spec, then sort each spec's items, then add them to the spec.
<StevenK> wgrant: Which means no load_referencing() :-(
<wgrant> StevenK: Sort afterwards.
<wgrant> Sorting it Python is fine when you need the whole set anyway.
<StevenK> wgrant: I see what you mean WRT the web UI, but I don't think I can tell the difference in a preload hook
<wgrant> StevenK: You can't, no. You need to use an argument.
<wgrant> Most of the methods that preload stuff like that have a kwarg to enable it.
<StevenK> wgrant: Right. We already have that
<StevenK> I'll check that the webui stuff turns off preload
<StevenK> wgrant: http://pastebin.ubuntu.com/5987337/
<wgrant> StevenK: Sounds reasonable.
<StevenK> wgrant: The diff is updated
<StevenK> But I suspect I need to rework the preload hook
<StevenK> We want to preload people always, but not workitems
<wgrant> Right
<wgrant> getPrecachedPersonsById or whatever it is has an option for each
<StevenK> Blah
<StevenK> visible_specifications and _valid_specifications are properties, and exported as such
<wgrant> StevenK: What's the issue with that?
<StevenK> wgrant: The browser code calls the same properties that the API does.
<StevenK> wgrant: http://pastebin.ubuntu.com/5987804/ is what I have, with the plan that the webui doesn't add prejoin_objects and the API does.
<wgrant> StevenK: Isn't linked_branches needed by the web UI?
<StevenK> wgrant: Hm, maybe
<StevenK> Then we have objects and extra_objects?
<StevenK> This is a horrible mess
<wgrant> StevenK: The others have a flag for each. need_branches, need_people, for example.
<StevenK> wgrant: Right, but I still don't see how I can seperate it out if it's called via properties
<wgrant> StevenK: You can't use the property.
<StevenK> Grump
<StevenK> Let me think
 * StevenK tries to work out how to combine functions
<wgrant> StevenK: Combine?
<StevenK> wgrant: If I have need_people, need_branches, need_workitems then I have three pre_iter_hooks to call
<wgrant> StevenK: Why not have one pre_iter_hook?
<StevenK> wgrant: Oh, with partial() ?
<wgrant> StevenK: with if
<StevenK> wgrant: I need to throw arguments at it, don't I?
<wgrant> StevenK: Closures
<StevenK> Bleh, nested functions
<StevenK> I think I've seen partial as an existing pattern
<wgrant> The existing stuff doesn't use partial
<StevenK> Aw
<wgrant> See prepopulate_person in _getPrecachedPersons
<wgrant> Pretty simple
<wgrant> That's a decorator rather than a pre_iter_hook, but you get the idea.
<StevenK> Yes
<wgrant> jelmer: Can you land the bzr-git thing?
<wgrant> Thanks for the reviews.
<jelmer> wgrant: you should be able to, I think
<wgrant> jelmer: I'm not in bzrish teams
<jelmer> wgrant: I'm sure that's fixable :-)
<jelmer> I guess I can land this one change; I'd like to avoid the impression that I'm actually maintaining it though.
<wgrant> jelmer: Who should I talk to about ~bzr membership?
<wgrant> Though it would be appreciated if you could land this one.
<jelmer> wgrant: sure, I can do that
<jelmer> wgrant: I think jam is probably the best person to ask about it
<wgrant> jelmer: Thanks, will do.
<wgrant> jelmer: Thanks!
<jelmer> thank you :)
#launchpad-dev 2013-08-16
<StevenK> wgrant: http://pastebin.ubuntu.com/5991234/
<wgrant> StevenK: Does it work?
<StevenK> wgrant: Total: 562 tests, 0 failures, 0 errors in 6 minutes 12.954 seconds.
<StevenK> That's -t test_product, which hits a lot, I'm kicking off -m blueprints now
<wgrant> Right
<StevenK> wgrant: Hm, do you think bug 1212703 is due to PTUJ ?
<_mup_> Bug #1212703: translations imports say wrong uploader <Launchpad itself:New> <https://launchpad.net/bugs/1212703>
<wgrant> StevenK: That isn't about packages, is it?
<StevenK> wgrant: I'm not sure, TBH
<wgrant> Those URLs are within a project.
<StevenK> Right, so unrelated.
<wgrant> I'd investigate the code to see how the uploader is determined.
<wgrant> It could, for example, be from the last-changed address in the pofile.
<wgrant> In some circumstances.
<StevenK> wgrant: I wasn't going to investigate it, I was just wondering if PTUJ might have broken it, that's all.
<wgrant> Ah
<StevenK> (Read as: I'm still fixing test failures)
<wgrant> StevenK: I'm about to land an upgrade to bzr 2.6.0, with new bzr-git, bzr-svn, dulwich, cscvs. I'll want to do a fair bit of QA on this, so if you have something ready that you want deployed soon, complain now :)
<StevenK> wgrant: Ooooh
<StevenK> wgrant: Any real fallout from the bzr upgrade?
<wgrant> cscvs was using a removed deprecated method
<wgrant> And LP needed a few lines of changes.
<wgrant> The main reason for this is the new bzr-git and dulwich, but I thought I might as well do the whole lot.
<StevenK> Right
<StevenK> wgrant: Nothing I'm really attached too
<StevenK> wgrant: But, microservices? :-)
<wgrant> Slow test suite is slow.
<StevenK> Bleh
<StevenK> I've deleted it from the property cache, why you not work :-(
<StevenK> Hm, except I haven't, it seems
 * StevenK peers at what is calling getDelta
<StevenK>  0.2s getDelta: get_property_cache(self).__dict__={'linked_branches': []}
<StevenK>  0.2s getDelta:
<StevenK>       get_property_cache(self).__dict__={'linked_branches': [], 'work_items': []}
<cjwatson> Has anyone managed to get the dogfood builders upgraded so we can QA the ABORTED status changes?
<wgrant> We were discussing that this morning.
<wgrant> But not yet.
<cjwatson> I'm at DebConf and my home ADSL has gone down, so I'm rather lacking in e-mail and IRC context at the moment.
<wgrant> Oh, fun.
<wgrant> (though this discussion was by voice, so you didn't miss IRC context)
<cjwatson> Right.  Just in case anyone was expecting me to know anything that lives in the external bits of my brain. :-)
<cjwatson> I'm on vacation next week, so if it takes until then, then somebody else will have to deal with unblocking the deployment pipeline.
<StevenK> cjwatson: wgrant is about to stop it up away
<StevenK> New bzr version
<cjwatson> Actually, hmm, I suppose that the proper QA for that branch is to make sure that it doesn't break with the *old* builders
<wgrant> We'd ideally upgrade some fraction of them
<wgrant> To test that it works with both
<jtv> Is anyone else having login trouble?
<jtv> I enter my username and password, and instead of the SSO page I get "Your page was stale."
<jtv> Deleted all launchpad cookies, reloaded everything... no difference.
<jtv> I'm finding some references on the intertubes, but they're fairly old.
<wgrant> jtv: You're probably not sending Referer to SSO
<jtv> No..  should I put it in a SASE?
<jtv> This is Chromium BTW...
<wgrant> Have you tried another browser, or a fresh profile?
<jtv> Firefox works fine.  It seems ScriptSafe is blocking referers â I have *.launchpad.net and *.ubuntu.com in my whitelist, but maybe the whitelist doesn't apply to referer blocking, or maybe there's another domain I need to enter?
<wgrant> login.launchpad.net and login.ubuntu.com are the two domains that matter.
<jtv> Then ScriptSafe is probably failing to honour the whitelist for click-through referer blocking.
<jtv> Grr..  Disabling referer blocking didn't do the trick.
<jtv> Ah, there's _also_ an option for sending a fake referer, which disables separately.
<jtv> wgrant: that worked!  \o/
<wgrant> jtv: Excellent
<wgrant> jelmer: Hah. Combining those two bug fixes breaks Google Code imports again. Google Code's HTTP server errors unless you support thin-pack, which we were accidentally before because of the negotiation bug in dulwich, which explains the ref deltas that revealed the bzr-git bug. Do you recall why thin-pack is disabled on fetch and push by bzr-git? It seems to work fine.
<lifeless> Maybe Jelmer can fix :)
<wgrant> It's trivial to fix, I'm just trying to work out why it's been disabled since 2009.
<lifeless> wgrant: I meant on the Google server side ;)
<wgrant> Oh, heh.
<wgrant> It sadly just seems to ISE in the face of any missing extensions :(
<jelmer> wgrant: hi
<jelmer> wgrant: I think the reason thin pack is disabled is that it can require expensive operations on the client side
<jelmer> so it's a CPU vs bandwidth thing
#launchpad-dev 2014-08-12
<cjwatson> wgrant: Do you think you'll have time to fix the copyPackage mail madness while I'm away?  I'm hoping not to get a pile of complaints on the 19th when we refresh ubuntu-rtm
<wgrant> cjwatson: Should do.
<cjwatson> Thanks.
#launchpad-dev 2015-08-10
<maozhou> The armhf package  building failed on builder of armhf. Build log: Error accessing Librarian: <urlopen error [Errno 111] Connection refused> . Why ?
<wgrant> maozhou: The builder can't talk to the librarian, apparently.
<wgrant> Is the librarian running?
<maozhou> On server or builder?
<wgrant> The librarian runs on a Launchpad server that is not a builder.
<maozhou> When  I run "ps -afx |grep librarian" , output message is " 3197 pts/0    Sl+    2:18  |       \_ /usr/bin/python2.7 -S bin/run -r librarian,sftp,forker,mailman,codebrowse,google-webservice,memcached,rabbitmq,txlongpoll -i development"
<wgrant> Can you access the librarian?
<maozhou> How to verify it?
<wgrant> Click a build log link.
<maozhou> Firfox can't access it .The message is " Firefox can't establish a connection to the server at launchpad.dev:58080."
<wgrant> Perhaps restart it.
<wgrant> Or otherwise check your firewalls.
<maozhou> But some other armhf builder is ok.
<wgrant> Then I would identify the differences between the ones that work and those that do not; this doesn't seem like a Launchpad problem.
<maozhou> ok
<rpadovani> cjwatson, o/ thanks for merging my MR :-) I'm on holiday this week, but when I'll come back home I'll try to work on another patch :-) Thanks for your time meanwhile
<wgrant> rpadovani: He's not here this week either, funnily enough.
<wgrant> rpadovani: Your changes all look good on production, as far as I can tell, but let me know if anything looks off.
<rpadovani> wgrant, okay, thanks :-)
<maozhou> The builder build package failed, the message is "dpkg-source: error: gpgv --keyring /usr/share/keyrings/ubuntu-archive-keyring.gpg", why ?
<wgrant> maozhou: You'll need to either debug it yourself or give us more details.
#launchpad-dev 2015-08-11
<wgrant> blr: I've got three MPs up on https://code.launchpad.net/launchpad/+activereviews when you've stopped having fun with juju and mojo.
<blr> yes, "fun"...
<blr> will have a look
<wgrant> I am adjusting squid-forwardproxy atm.
<blr> wgrant: ah, what in particular?
<wgrant> blr: More flexible ACLs.
<blr> wgrant: are you touching auth_list?
<wgrant> blr: It is likely.
<wgrant> I'm trying to work out how to salvage it.
<wgrant> That doesn't go near your work, does it?
<blr> yeah, it isn't great by any means.
<blr> no that's fine, might need to update the mojo spec though.
<blr> wgrant: did you have a look at the auth_param changes btw?
<wgrant> blr: Ah, no, I didn't see an MP.
 * wgrant scratches head
<wgrant> "This charm requires the following relation settings from clients:
<wgrant> ip: service ip address
<wgrant> port: service port
<wgrant> sitenames: space-delimited list of sites to whitelist
<wgrant> "
<wgrant> What possible use could the service port be to the proxy?
<wgrant> blr: https://code.launchpad.net/~wgrant/charms/precise/squid-forwardproxy/auth_list-deny/+merge/267626
<blr> wgrant: looks like you've already merged it, sorry - certainly an improvement, particularly handling the inverse is nice.
<blr> wgrant: wrt to auth_param, without any special casing we would need a dict without nesting similar to {'scheme': 'basic', 'children': '0 startup=N idle=N'} etc.
<blr> was inclined to have all of the params mapped to keys, but probably just as clear that way
<wgrant> blr: I haven't merged it yet, and I realised a case last night that I'm not sure I can be bothered handling in a backward-compatible way, so it can probably wait for the more sensible redesign.
<wgrant> Specifically, there's no way to specify the order of ACL elements within an http_access line, which can be relevant when you're relying on short-circuiting for performance or to avoid unnecessary auth prompts.
<blr> wgrant: any thoughts on the auth_param stuff above? ^ will that suffice?
<wgrant> blr: Hm, perhaps have a special case that if the "children" key is a dict then it gets turned into a string?
<wgrant> So {"scheme": "basic", "children": "11", "utf8": "off"} and {"scheme": "basic", "children": {"number": 11}, "utf8": "off"} would be equivalent.
<wgrant> I suppose turning bools into strs would also make sense.
<wgrant> So you can still specify anything you want using strings, but there are also niceties for certain keys.
<blr> right
<wgrant> Does that make sense to you?
<wgrant> I think it's the least jarring approach.
<blr> yep, that seems reasonable
#launchpad-dev 2015-08-12
<mwhudson_> wgrant: could you devirt and all-arch-enable https://launchpad.net/~mwhudson/+archive/ubuntu/go15-rc-tests pls?
<blr> wgrant: could you have a look at the MP, I'm not sure I suceeded in reducing code (required the addition of a jinja2 test, but the configuration is more straight forward as a result.
<wgrant> blr: Oh, I would have just done the processing in Python, but that works too.
<wgrant> blr: The template also means the dict has to have number, startup and idle members, and doesn't support concurrency or queue-size.
<wgrant> Doing it Python would give you more flexibility to just flatten whatever keys are in the children dict.
<blr> wgrant: hmm true.
<lifeless> since its webby, you could use handlebars / pybars :)
<wgrant> blr: Could you please review the auth_list branch?
<blr> wgrant: ah did you resolve the issue you mentioned earlier?
<wgrant> blr: Oh, no, didn't you see the carpet over it?
<wgrant> It's not fixable in an attractive and backward compatible way, and it isn't a problem for either of our current use cases.
<wgrant> auth_list needs to replaced with more generic acls and http_access options.
<wgrant> But this branch doesn't make deprecating auth_list any harder when that eventually becomes necessary, so I think we should proceed with it.
<wgrant> blr: Thanks.
<blr> wgrant: have another interview tomorrow, but hopefully shouldn't run into our LP meeting.
<wgrant> blr: Sure, let me know.
<blr> morning
#launchpad-dev 2015-08-13
<blr> wgrant: a css improvement for diffs here (~blr/launchpad/diff-code-select-bug-1483925), making the lineno column unselectable. I think in almost every case you would _not_ want the line numbers when copy/pasting, however potentially you might want the diff markers?
<blr> could render them with the same style, making only the code selectable.
<blr> haven't forgotten your MPs btw, lost a bit of time to lxc this afternoon
<wgrant> Yep
<blr> I think preserving the diff markers is the correct thing to do.
<blr> wgrant: did the charm changes look sane?
<blr> certainly makes the template simpler.
<wgrant> blr: Let me see.
<wgrant> blr: You have a conflict, otherwise that looks good.
<blr> ah thanks, will fix that up.
<wgrant> blr: I agree that the diff markers should be preserved.
<wgrant> blr: What is the behaviour in side-by-side mode?
<blr> wgrant: are you aware of any mechanism for limiting text selection to a given column? Firefox has support for that, but it seems strangely absent in other browsers.
<wgrant> blr: Oh, I didn't know even Firefox supported it.
<wgrant> blr: In loggerhead we don't use a table, and lay out the DOM such that the line numbers are totally separate.
<blr> wgrant: yep, seems like a useful feature, pity chrome doesn't have something similar.. don't think there's any way to emulate that with css
<blr> wgrant: why is logout() required, is that not handled by the admin_logged_in() context?
<blr> err sorry context, looking at the bugtask attachment fix
<wgrant> blr: The *_logged_in context managers temporarily switch out the interaction, but they restore the old one afterwards.
<blr> ah I see
<wgrant> So you end up with an anonymous interaction, rather than the no interaction at all required by webservice tests.
<blr> that's good to know
<blr> testtools matchers are handy, I should make use of those more often.
<wgrant> They are marvellous once you get the hang of them.
#launchpad-dev 2015-08-14
<blr> wgrant: would it worth be including the new head in the payload?
<blr> and maybe a timestamp?
<blr> commented on the MP
<wgrant> blr: Timestamp maybe, but all the ref changes are included.
<wgrant> Already.
<blr> wgrant: not sure I understand the change to the signature of WebhookDeliveryJob.create
<wgrant> blr: Adding delivery_id and event_type so they can be passed down to create_request to be included in the request headers.
<blr> ah I see, thanks
#launchpad-dev 2016-08-16
<bigjools> Someone remind my tired brain, can a non-team-admin create a team PPA or is it restricted to team admins?
<cjwatson> bigjools: Team admins only.  (You need launchpad.Edit on the PPA owner.)
#launchpad-dev 2016-08-17
<bigjools> cjwatson: thanks for the reminder! I've been away too long ...
<bigjools> This is amusing, it's not been updated for 4 years https://dev.launchpad.net/LEP
#launchpad-dev 2019-08-12
<SpecialK|Canon> cjwatson: well http://blog.launchpad.net/general/launchpad-news-march-2019-july-2019 was a nice read, cheers!
<cjwatson> SpecialK|Canon: good!  in theory I do them monthly, in practice uhhh
<SpecialK|Canon> cjwatson: can I help?
<cjwatson> possibly!  next month though :)
<SpecialK|Canon> Sure :)
<cjwatson> I don't have a single source of information so it tends to be going through commit logs of everything I remember us touching ...
<cjwatson> filtering out things that are boring, coalescing things that are all part of the same basic change, and trying to roughly categorise the rest
<SpecialK|Canon> Nodnod
<SpecialK|Canon> What's a Pillar? Attempting to search dev.lp for it just takes me to https://dev.launchpad.net/Registry/PillarAliases which presupposes my knowing the definition...
<cjwatson> SpecialK|Canon: See lib/lp/registry/interfaces/pillar.py
<cjwatson> "An object that might be a project, a project group, or a distribution."
<cjwatson> SpecialK|Canon: The reason to have this as a concept is that the set of pillars is the set of non-hardcoded names that can be the first URL segment under https://launchpad.net/
<SpecialK|Canon> cjwatson: brilliant, thanks
