#launchpad-dev 2009-07-27
<mars> thumper, ping, what should I do for https://answers.edge.launchpad.net/launchpad/+question/77866 ?
<mars> is it OK to just nuke a vcs import like that?
<mwhudson> if the user asks, yes, generally
<mwhudson> i guess you might consider suspending, renaming and marking it abandoned
 * mwhudson goes for lunch
<mars> thumper, ping, how do you assign a VCS import as a project's trunk/
<mars> or any branch, for that matter...
<mars> in reference to https://answers.edge.launchpad.net/launchpad/+question/78014
<mwhudson> mars: you need to be a project maintainer
<AdamDV> Anyone here?
<spiv> Yep.
<AdamDV> Any diagnosis on:
<AdamDV> make: *** No rule to make target `install'.  Stop.
<AdamDV> ERROR: Unable to install apache config appropriately
<AdamDV> Any ideas spiv?
<mwhudson> i think we need more context
<AdamDV> I just ran ./rocketfuel-setup
<AdamDV> I shall delete lp-branches and try again
<AdamDV> Hmm
<mwhudson> probably not
<AdamDV> Should the progress bar be stuck at the middle?
<AdamDV> Its right now like this:
<mwhudson> oh
<mwhudson> hm
<AdamDV> [#########|          ]  20274KB    78KB/s | Fetching revisions:Inserting stream
<mwhudson> yeah, it might get stuck for a while like that
<mwhudson> it has to download ~150 megs
<AdamDV> Its been like that for at least 5 minutes.
<AdamDV> the KB meter is advancing, but the ###'s arent
<AdamDV> Which I assume is fine
<wgrant> bzr likes doing that.
<wgrant> I let it sit like that for nearly 5 hours a week ago, and it finished fine.
<wgrant> Is LP still a hot product?
<AdamDV> Yes
<mwhudson> wgrant: yeah
<wgrant> mwhudson: :(
<AdamDV> Has anyone actually gone through the rocketfuel setup here?
<AdamDV> Sucessfully?
<ajmitch> yes, a number of us have
<ajmitch> I haven't tried on a clean jaunty install, which is the most likely to get a good result I think
<wgrant> AdamDV: Worked fine for me on Jaunty.
<wgrant> It just took ... a while.
<wgrant> But it was the first night, and I am in .au.
<AdamDV> wgrant: how long?
 * ajmitch ended up leaving it to branch at night
<wgrant> AdamDV: Um, it took about 5 hours all up, but that was with a bug or two in rocketfuel-setup that made it take far too long.
<wgrant> The actual RF branch checkout took 3-ish hours.
<wgrant> And downloaded 280MB.
<AdamDV> Bug fixed?
<wgrant> (that's by the counter bzr gives you)
<wgrant> Yes.
<wgrant> The bug was fixed.
<wgrant> Otherwise it would have taken many hours longer.
<AdamDV> Ah
<AdamDV> Man, can't I download the branch anywhere?
<wgrant> bzr get lp:launchpad
<wgrant> But that's what it's doing now.
<wgrant> It will take a while.
<ajmitch> didn't the wiki refer to a tarball of the branch that may be useful?
<wgrant> That's true.
#launchpad-dev 2009-07-28
<gmb> Hurrah for /++form++ links. Sadly, in this case, I can't use one.
<wgrant> Why does DistroSeries.lucilleconfig still exist? Shouldn't ComponentSelection cover that?
<bigjools> wgrant: legacy.  it needs to die
<wgrant> bigjools: Ah, good.
<gmb> Argh. The bloody branch picker is horrible, horrible, horrible.
<deryck> Good morning, all.
<mrevell> yo deryck
<beuno> good morning
<noodles775> hey beuno !
<beuno> I slept today!  :)
<noodles775> Good to hear!
<gmb> deryck: Damn, I can see why firebug's annoying you now.
<wgrant> It's not just me, then?
<deryck> gmb, yeah, it's beyond painful to me now.  I can't even use it without popping it out into a new window now.
<gmb> wgrant: I don't think you've done all that much to annoy deryck. Keep trying, though.
<gmb> Oh, wait.
<deryck> heh
<wgrant> Heh.
<deryck> I'm actually not easily annoyed... but firebug and I are on not-so-friendly terms these days.
<gmb> deryck: Do you know if there are any good notes about using the JS webservice client? It's frustrating me to death.
<deryck> gmb, I don't know of any notes, sorry.  I just grep around the js files we have for prior art.
<gmb> Rats.
<gmb> deryck: Oh well, thanks anyway. Maybe I need food before continuing. Yes, that seems likely.
 * gmb -> lunch
<mrevell> I've just posted an interview with the people from Open Mind, a MIT project to build a database of common sense concepts. They're using LP. Some interesting stuff -- http://blog.launchpad.net/projects/open-mind-and-launchpad
<deryck> oh happy freakin' day I fixed firebug for me.
<deryck> gmb, uninstall firebug, close FF, grep your profile directory for firebug entries (mostly locatestore.rdf and prefs.js), delete those entries and then reinstall firebug.  Firebug behaves again!
<bac> mrevell-lunch: ping me when you're back from lunch
<beuno> mrevell-lunch, are we on for our call in 10?
<deryck> leonardr, ping.
<leonardr> hi deryck
<deryck> leonardr, hi.  I have a question about the js web service client and trying to get a result back in html...
<leonardr> ok
<deryck> leonardr, I'm trying something like: https://pastebin.canonical.com/20439/
<deryck> leonardr, am I on track here, or is there a better way?
<leonardr> deryck, what is the PATCHplugin? i've never heard of it
<leonardr> the code looks fine but i just never heard of that PATCHplugin
<leonardr> the way you get an xhtml representation is to specify 'accept', as you did
<deryck> leonardr, PATCHplugin is in lib/canonical/launchpad/javascript/client/client.js and comment says: * This plugin overrides the widget _saveData method to update the * underlying model object using a PATCH call.
<leonardr> oh, ok. so it's a way of making the web service the destination of widget actions
<deryck> leonardr, right.  So this looks like the right path I'm headed down then?
<leonardr> deryck, yes, it looks like it
<deryck> leonardr, excellent.  thanks for the help!
<leonardr> np
<gmb> deryck: Ah! That's... annoying but useful. Thanks.
<deryck> gmb, yeah, you would think it could blow away previous defaults or somehow be backwards compatible.
<sinzui> bac, Edwin, salgado: standup in 2 minutes
<intellectronica> leonardr: i'm having an interesting problem using the webservice. it seems it can't mix between api.launchpad.net/beta and launchpad.net/api/beta url. if i post to and using the other in a value it complains
<intellectronica> leonardr: i've got a workaround, but i think it would make sense to allow both, no?
<leonardr> intellectronica, are you using an ajax client?
<intellectronica> leonardr: indeed
<intellectronica> oh, of course, domain restrictions!
<leonardr> actually, it might not be because of domain restrictions, now that i think about it
<intellectronica> still, i'm not trying to access a different domain. i just pass a uri as a value
<leonardr> i think the parser assumes incoming urls use the same host as the host of the current request
<leonardr> that'd be pretty hard to change; how annoying is this for you?
<intellectronica> leonardr: it's annoying in that i have to write code especially to convert the result of canonical_url (which uses the api domain) to the format which uses the /api root. it seems wrong to have that hard coded
<intellectronica> as i said, it works now, so i can leave it with an XXX, but i think that it would be good if in the future we handled that
<leonardr> intellectronica: i think bjornt has fixed the underlying problem
<intellectronica> leonardr: even in the absence of a webservice request?
<leonardr> intellectronica: ok, let's back up
<leonardr> where are you getting the data that you have to convert?
<intellectronica> the problem i have is that i do the formatting in a normal web request
<intellectronica> leonardr: while processing a normal browser request, i take an object (from a vocabulary) and i want to get the api url for it, which i will use when issuing a PATCH request from the AJAX client
<leonardr> ok, you are writing code in launchpad
<leonardr> i thought you were writing client code
<intellectronica> yup
<intellectronica> leonardr: well, i'm doing both. the client code wants an api url in the same domain as the page
<intellectronica> but i'm preparing this value in advance when rendering the page
<leonardr> intellectronica, what bjornt did was change canonical_url so that if you did not pass in a request object, it would give you the object's website url, not the web service url
<intellectronica> leonardr: right, so that won't solve my problem
<leonardr> but that won't help you. you need to get the object's web service url for the web service as accessed through the website
<intellectronica> leonardr: what i have to do now is get the website url, parse it, inject '/api/beta' and unparse
<intellectronica> leonardr: maybe all i need to do is extract this to a function we can use everywhere. i don't know if it makes sense to overload canonical_url with more stuff
<leonardr> intellectronica: in theory there's some request object you could create and pass into canonical_url that would make canonical_url do what you want
<leonardr> canonical_url already generates those urls under some circumstances
<intellectronica> leonardr: ok, that may be a better way to implement that, but i still don't think i should have to do that every time i need to generate a url
<intellectronica> leonardr: also, it seems a bit wrong to create a request object just for that purpose
<leonardr> intellectronica, we create request objects all the time for similar purposes
<leonardr> changing canonical_url is a huge task
<intellectronica> fair enough. do you maybe remember a place where we do that?
<intellectronica> leonardr: i agree. i think i'll just create a new helper function, webservice_canonical_url() or something like that
<leonardr> intellectronica: check out bjornt's code. you can do the same thing in your helper function, but also check canonical_url to see what the request has to look like for that kind of url to be generated
<intellectronica> leonardr: cool, thanks
<BjornT> intellectronica: it might work if you simply do: canonical_url(ob, request=IWebServiceClientRequest(request))
<BjornT> intellectronica: but hard to say without looking at your code
<intellectronica> BjornT: yes, that seems to work just fine. thanks!
<gmb> Gah. Did I mention that I hate Javascript?
<mars> :)
<maxb> Does anyone like Javascript? :-)
<kfogel> mrevell: ready to skype?
<mrevell> Hi kfogel, almost, just making a cup of tea
<kfogel> mrevell: no rush, I'm on skype now
<kfogel> mrevell: meaning, available when you're ready, but take your time
<gmb> Does anyone have any idea under what circumstances a named_post would just sort of vanish into the ether when called from Javascript? Neither success nor failure callbacks are called.
<gmb> Ah, wait.
<gmb> If I actually read the output from the server, that would help.
<rockstar> morning
<noodles775_> beuno: It'd be great to be able to find out exactly what the purpose and use-cases of a page is on LP - documenting that info in some permanent location.
<noodles775_> beuno: I think mars mentioned something along those lines the other week...
<beuno> noodles775_, yes
<noodles775_> beuno: but I've created https://dev.launchpad.net/SoyuzBuildersIndexPage
<noodles775_> as an example, and will fill it out as I get info, but is there anything you'd add (I'm about to create a second one for another page)
<beuno> noodles775_, on calls, but will look at it afer them
<noodles775_> beuno: yep, no rush. Thanks.
 * rockstar is glad to have 4 cores, since make schema is chewing on an entire core
<sinzui> beuno: I would like your opinion of the screencaps for bug 399010.
<ubot3> Malone bug 399010 in launchpad-registry "Update simple edit pages to base-layout" [High,In progress] https://launchpad.net/bugs/399010
<mup> Bug #399010: Update simple edit pages to base-layout <story-ui-3> <Launchpad Registry:In Progress by sinzui> <https://launchpad.net/bugs/399010>
<mup> Bug #399010: Update simple edit pages to base-layout <story-ui-3> <Launchpad Registry:In Progress by sinzui> <https://launchpad.net/bugs/399010>
<mup> Bug #399010: Update simple edit pages to base-layout <story-ui-3> <Launchpad Registry:In Progress by sinzui> <https://launchpad.net/bugs/399010>
 * sinzui now hacks on project page
<rockstar> Looks like we need to either kill ubot3 or mup...
<gmb> rockstar: Better still, make them fight until one dies.
<sinzui> two bots enter, one bot leaves
<sinzui> I suspect mup is cheating and may in fact be 3 bots
 * gmb EODs
<beuno> sinzui, finishing lunch, and will look
<abentley> rockstar: chat?
<rockstar> abentley, sure.  One sec.
<abentley> rockstar: https://launchpad.canonical.com/DeveloperDocumentation/TemplateCodeReuse
<rockstar> abentley, actually, the more I think about it, the more it would seem to follow current conventions if it was a link to a single submit button form value.
<rockstar> abentley, although I am not qualified to make UI decisions.
 * rockstar hates the look of HTML buttons
<abentley> rockstar: I think following conventions is good, up to a point.  I think that having people go to another page just to click a submit button is past that point.
<rockstar> Yes, this is true.  I'm pretty torn.
<abentley> You could almost justify it if you were asking for confirmation, because claiming a review can't be undone.
<beuno> sinzui,
<beuno> are you up for a massive challenge?
<rockstar> abentley, ah, yes, that would actually make sense.
<sinzui> Dude. I just discovered that the new heading and the title edit-widget are incompatible. I am already in a challenge
<beuno> sinzui, heh, ok
<beuno> I'll try and tackle it then
<beuno> we need to fix our freakin forms
<sinzui> beuno: but I am available
<beuno> this right-aligned bullshit has to stop
<rockstar> beuno, AMEN!
<beuno> I've tried to train my eyes to ignore it, but I can't
<sinzui> beuno. Yeah. There are some near infinite length fields too
<beuno> sinzui, I'll address
<beuno> I'll do it NOW
<sinzui> beuno: a subtle change to launchpad-form-macros will change everything, as too might the css rule
<beuno> sinzui, more from my ourage soon
<beuno> I was looking through your screencaps
<beuno> but that issue lept coming up
<beuno> sinzui, the template changes look
<beuno> fine
<beuno> there are a lot of peripheral issues to solve
<sinzui> beuno: I understand. That is why the screencaps are repetative...so it is obvious there is something else we need to do to land my branch
<beuno> :)
<beuno> sinzui, there's something else
<beuno> about a change we made to edit/add/remove icons
<beuno> the rule should be: if you can edit/remove an object, the icon should be on it's right
<beuno> if it's an action, on the left
<beuno> is there any way we can change that?
<beuno> I don't like the result of having the add/remove on the right of action links
<sinzui> beuno: How do we know the link is an action?
<beuno> sinzui, my guess is that the only way is to specify it?
<beuno> by default it's an action, but a parameter modifies that?
<sinzui> Well in th screencaps, what is wrong with the links?
<sinzui> which ones are wrong
<beuno> Related actions
<sinzui> beuno: you are editing the project in all cases
<beuno> sinzui, right, but they are action links
<beuno> what I mean by object, is the actual's object name
<beuno> Project name (i)
<beuno> (I) Edit something something
<beuno> in the first case, the important information is the object's name, and secondary is the fact you can edit it
<beuno> and in the second case, the important information is that you can edit something
<sinzui> beuno: If the icon must follow something else in the page, then we use fmt:icon, and we make fmt:icon always render the with the icon on the left
<beuno> sinzui, and for editable objects?
<sinzui> So the rule is: developers may only use fmt:icon  when they are placing a link to edit the "thing" that just rendered
<sinzui> is editable objects the same as "thing" that just rendered?
<beuno> yes, but that means it should be on the right
<beuno> I think add will *always* be on the left, for example
<beuno> because it has to tell you what it adds
<sinzui> beuno: I think the rule then is the link text can never be rendered before the icon. The icon may be rendered after the object's text/representation
<beuno> but edit and remove placed next to something is clear enough
<beuno> yes to that second statement
<beuno> rockstar, have you heard about BjornT's super inline bug commenting branch?
<sinzui> beuno: in https://bugs.edge.launchpad.net/launchpad-registry/+bug/399010, I see "Unsubscribe (-)".
<mup> Bug #399010: Update simple edit pages to base-layout <story-ui-3> <Launchpad Registry:In Progress by sinzui> <https://launchpad.net/bugs/399010>
<ubot3> Malone bug 399010 in launchpad-registry "Update simple edit pages to base-layout" [High,In progress]
<mup> Bug #399010: Update simple edit pages to base-layout <story-ui-3> <Launchpad Registry:In Progress by sinzui> <https://launchpad.net/bugs/399010>
<beuno> rockstar, I think you should steal it for reviews as soon as it lands
<beuno> sinzui, right. I think that looks wrong.
<beuno> it should be "(-) Unsubscribe"
<sinzui> beuno: I think by your rule (fuck you mup) the link should be...okay we agree
<beuno> :)
<beuno> Martin Albisetti (-)
<sinzui> beuno: I think I can fix this in a few deft lines of code.
<beuno> unsubscribes me
<beuno> sinzui, you will elevate your status from personal heroe to personal god
<sinzui> If only I had physique of a code instead of an deskbound hacker
<sinzui> wow, how did "code" get into that sentence
<sinzui> oh well.
 * beuno wonders what he meant
<sinzui> s/code/god/
<beuno> :)
<beuno> BIG typo there
<EdwinGrubbs> sinzui: the SourcePackageNameVocabulary is in registry/model/sourcepackagename.py instead of registry/vocabularies.py. Is this part of the reorg or just wrong?
<sinzui> I think it was wrong and was not automatically fixed by the migration script
<rockstar> beuno, sounds good.  I have a backlog of branches that need to land first, but I'll keep it in mind.
<beuno> bigjools, mockup is looking better
<beuno> although seeing it with data it makes me realize some of my thoughts where bogus
<beuno> sinzui, I went through an intensive review of navigation with the UX team and Mark today
<beuno> and we're looking good
<beuno> I should get a final-ish mockup back tomorrow
<sinzui> rock
<beuno> which is *very* similar to the last one you saw
<rockstar> Would someone like to do a JS review for me?
<sinzui> beuno: This is the fix for the modify icons: https://pastebin.canonical.com/20476/ I replaces "sprite-after" with "sprint modify". We do not have a .modify class. If you need to act just on those links, you now have a hook
<sinzui> s/"sprint modify"/"sprite modify"/
<beuno> sinzui, rock
<beuno> thanks
<beuno> deryck may go crazy
<sinzui> This will land when pqm opens
<beuno> thanks sinzui
 * deryck looks at scrollback, smelling icon changes again
<deryck> beuno, why will I go crazy?  Because of changing the icon positioning again? :)
<sinzui> deryck: only if you are not using the approved way of making links. fmt:link JDRT
 * beuno hides
<deryck> sinzui, fmt:link doesn't exist in js. :)
<sinzui> deryck: fmt:link-icon and fmt:icon-link will also work because They are just aliases for fmt:link
<sinzui> deryck: if you used sprite-after, you will need to change it to sprite modify.
<deryck> sinzui, yeah, I don't think it's going to be a pain.  Was just going to give beuno a hard time about it. :)
 * beuno unhides!
<beuno> aha!
<beuno> :)
<beuno> I promise, I'm standarizing on this rule
<beuno> sinzui, how crazy do you think I am if I drop the use of tables from forms?
<rockstar> deryck, I think you might like the branch I just sent you.  I'm working on devising a way of pulling markup out of js and back to tal again.
<beuno> I will have to spend a year of my life fixing tests, right?
<deryck> rockstar, oh cool
<sinzui> beuno: you will cry. the problem is larger than sprites
<deryck> rockstar, did you ever branch subscriptions?
<beuno> crap
<deryck> ever do, that is.
<sinzui> beuno: there are a lot of widgets that render as tr and td combos because they know all forms are tables
<rockstar> deryck, I haven't finished it, but I will need to make LOTS of changes to it.  It's going to be more complicated than bug subscriptions, because our subscriptions have state.
<deryck> rockstar, right.
<beuno> ok, well, I'm going to ask to get this for my birthday then
<sinzui> beuno: they also assume the table is 2 column. I have seen BAD thing happen when they do not mix well
<deryck> rockstar, I do have nicer Subscriber and Subscription objects in js now, which you may be better able to re-use.  If you haven't seen those yet.
<rockstar> deryck, I saw that.
<deryck> doesn't help the state difference, but handling users is slightly better.
<beuno> sinzui, I could cheat and make everything colspan=2  :)
<beuno> especially now that flacoste is on vacations and can't see me
<sinzui> beuno: hey! that is my cheat. I used it last release
<beuno> fantastic, I know have someone to blame
<sinzui> beuno: use the change password view to verify our hacks work
 * beuno stabs stuff
<beuno> sinzui, the password change form seems to do something funky
<sinzui> beuno: it does!
<beuno> why?!
<sinzui> beuno: solve that form and most of the widget weirdness will also be solved
<beuno> and where do I stab it?
<beuno> I mean
<beuno> I changed the forms macros
<beuno> and the first field is fixed
<beuno> but the next two remain the same
<beuno> they have th's out of nowhere!
<sinzui> beuno: The password widget is inserting it's own markup. many of our widgets do
 * beuno bangs head against table
<beuno> sinzui, so I should dive into python?
<beuno> why does it *just* do that for 2 fields?  when all 3 of them are exactly the same?
<sinzui> beuno: EdwinGrubbs discovered that the widgets in that form never align. We decided that setting the width of one column will give the illusion on good design
<sinzui> beuno: I think you need to look at the HTML templates used by each widget.
<beuno> *sigh*
 * beuno removes infinite length for input boxes
 * sinzui high five's beuno
<beuno> sinzui, I will want you to review this branch
<sinzui> beuno: I think these are the problematic controls: https://pastebin.canonical.com/20478/
<sinzui> sure
<beuno> I want to make non-optional labels bold
<beuno> thoughts?
<beuno> oh, THANK YOU for that list
<sinzui> beuno: I support every effort to fix the madness. I looked at the issue last year and walked away
<beuno> does anyone remember the large ec2 instance's name?
<beuno> -i x1.large?
<beuno> gary_poster, ^
<beuno> (and, can we make that the default?)
<gary_poster> beuno do ec2test --help
<beuno> ah  :)
<beuno> I did, but didn't read
<gary_poster> :-)
<beuno> thanks gary_poster
<gary_poster> np
<beuno> any reason why that's not the default?
<beuno> oh, it says it does now
<maxb> gary_poster: Hi - Your name has been mentioned in connection with Launchpad + Python 2.5 a bit - are there any bugs / branches that I, being interested in that, might want to keep an eye on?
<gary_poster> maxb: hi.  When I get back to this branch and get it landed, we'll have the infrastructure for others to start helping (by moving to newer versions of dependencies, for instance): https://edge.launchpad.net/~gary/launchpad/zbuildout
<maxb> Can you un-private that branch, if you're at liberty to do so?
<gary_poster> maxb: oh, sure, sorry.  haven't done that.  beuno (since you tagged me ;-) ), I either ask a losa to do that, or subscribe the launchpad public team, or just push another new branch up, right?
<maxb> There is no edit icon for you next to "This branch is private" on https://edge.launchpad.net/~gary/launchpad/zbuildout ?
<beuno> yes
<maxb> (I'm randomly speculating that it should work the same way as private bugs, since I've never seen a private branch)
<beuno> subscribe people
<gary_poster> maxb: no
<gary_poster> beuno ack thanks
<gary_poster> maxb: kinda busy, I'll just add you for now.  what's your account?
<maxb> Just maxb, but no rush, it can wait until you can do it properly
<maxb> sidnei already pushed a public copy a couple of days ago
<gary_poster> maxb: ok cool.  I plan to do it before EOD.  thanks for understanding.  ping me anytime you like
 * gmb wishes for a pqm-submit --do-what-i-mean option.
<beuno> fsck
<beuno> I forgot to use --headless
<beuno> gary_poster, do I have any other options than killing it?
<gary_poster> beuno: no, sorry
<beuno> no problem, stupidity has a price
<beuno> in this case, probably about 2usd
<gmb> \0/ wgrant becomes the first community contributor to have code landed on devel. Hurrah, etc.
<gmb> (at least I think he does)
<kfogel> maxb: hey, I noticed you edited the dev.launchpad.net wiki (thanks).  Was it open when you first tried, or did you have to get someone to remove the ACL?
<thumper> morning
<beuno> hey thumper
<mars> morning thumper
<beuno> I want to talk to you when you're caffeinated  :)
<thumper> I see that pqm is open again
 * thumper submits some branches
 * mars was just being social
<thumper> beuno: what was your bug number for your branch listing oopses?
<thumper> hi mars
<thumper> beuno: nm
<beuno> kai
<thumper> beuno: LP looks very sucky in FF 2 on Windows :(
<beuno> does it?
<beuno> I'll have to take a look and see why
<thumper> yep
<thumper> sprits :(
<thumper> sprites
<beuno> really?  they should work fine on FF2, there's no black magic in them
<thumper> beuno: we can talk now if you like
<thumper> beuno: I have 20 minutes before the standup
<beuno> thumper, sounds good
 * beuno headphoneses
<beuno> thumper, ready
<thumper> beuno: my skype is having trouble finding you
<thumper> beuno: try to call me
<maxb> kfogel: I waited until I noticed an Edit link. I understand it's generally open now.
<maxb> I see the front page is locked - presumably to discourage stupidity
 * ajmitch wonders why rocketfuel-get is failing now as it tries to get a bzr lock
<ajmitch> more specifically, it's only ~/launchpad/lp-sourcedeps//download-cache that's failing to update, saying that it's a readonly transport
<wgrant> gmb: Thanks!
<EdwinGrubbs> sinzui: we kinda forgot about the weekly meeting.
<sinzui> EdwinGrubbs: yeah. We can talk sometime tomorrow about the pickker work
<ajmitch> wgrant: as a quick check, are you able to update ~/launchpad/lp-sourcedeps/download-cache ? It doesn't want to update with bzr+ssh for me
<wgrant> ajmitch: WFM
<ajmitch> bzr+ssh for you?
<wgrant> Yep.
<ajmitch> interesting, I just get:
<ajmitch> bzr: ERROR: Cannot lock LockDir(lp-45197200:///~launchpad/lp-source-dependencies/trunk/.bzr/branchlock): Transport operation not possible: readonly transport
<wgrant> What are you trying to do?
<ajmitch> Run rocketfuel-get & not have it fail
<wgrant> What bzr action gives that error?
<ajmitch> bzr up "$LP_DOWNLOAD_CACHE_PATH"
<ajmitch> which is ~/launchpad/lp-sourcedeps/download-cache
<ajmitch> nothing out of the ordinary there
<wgrant> Very odd.
<ajmitch> The only thing I can see is that it's a checkout, not a branch
<beuno> thumper, I'm filing the "bzr push lp:project" bug anyway, since I can't find it
<beuno> unless you can  :)
<thumper> beuno: ok
<thumper> beuno: we could always dupe it anyway
<beuno> yes, free kama
<beuno> karma as well
<beuno> aha
<beuno> bug 181401
<ubot3> Malone bug 181401 in launchpad-code "should be able to push to lp:project or lp:project/series to make a new branch" [Medium,Triaged] https://launchpad.net/bugs/181401
<mup> Bug #181401: should be able to push to lp:project or lp:project/series to make a new branch <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/181401>
<mup> Bug #181401: should be able to push to lp:project or lp:project/series to make a new branch <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/181401>
<mup> Bug #181401: should be able to push to lp:project or lp:project/series to make a new branch <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/181401>
<beuno> thank you mup
<rockstar> Who is in charge of ubot3?  Can we get it disabled?
<beuno> done  :)
<rockstar> Thank you beuno
 * wgrant had better get going on another branch before bigjools clears the queue tonight.
<maxb> mup would seem to have a bug too, since it's replying three times when only a person and ubot3 said anything
<wgrant> maxb: The third was probably from the URL in ubot3's reply.
<maxb> hmm
<wgrant> It's reasonable to look for multiple bugs, so I don't think that's a bug.
<beuno> it's mentioned 3 times
<beuno> once by me
<beuno> and twice by ubot3
<wgrant> At least ubottu and derivatives have a limit on how often they'll detect a mention of a bug.
<wgrant> I can't see how badly I killed buildbot :(
<ajmitch> Do you suspect that you may have murdered it?
<wgrant> It's the first community contribution, and it's by me, so something has to go wrong.
<thumper> wgrant: what do you think has happened?
<thumper> wgrant: I don't see anything wrong
<wgrant> thumper: Ah, good.
<beuno> thumper, any idea how to conditionally display a specific CSS class in TAL?
<beuno> as in, always show the tag
<thumper> beuno: yes
<beuno> but on a condition, display a class
<mwhudson> is there a reason the buildbot waterfall is still private btw?
 * beuno waits to be enlightened
<mars> beuno, eeew, not fun to do in TAL
<thumper> mwhudson: not any more
<thumper> beuno: well...
<thumper> beuno: it depends
<wgrant> Particularly in Python 2.4...
<beuno> I know it's something like  $condition|something/something
<thumper> beuno: you want to use tal:attributes="class ..."
<beuno> yes, I got that far
<thumper> beuno: got a concrete example for me?
<beuno> but blah|blah only returns True/False
<beuno> yes
<beuno> one sec
<mwhudson> beuno: put it on the view
<mwhudson> i think returning None will mean not including the attribute at all
<beuno>           <label tal:attributes="for widget/name;
<beuno>                                  class show_optional|I-want-a-class-here"
<beuno>                  tal:content="string:${widget/label}:">Label</label>
<beuno> the thing is
<mwhudson> or a python: expression, but ewww
<beuno> this is the forms macros
<beuno> so I could put it in it's view
<beuno> but I ahve this right above it:
<beuno>         <tal:block tal:condition="display_label|widget/display_label|python:True">
<thumper> as mwhudson said, a python tal expression is one way
<thumper> or put the classname onto the view itself
<mwhudson> beuno: how about you tell us what you're actually trying to do?
<mwhudson> (a line that never gets old!)
<beuno> ok
<beuno> I want for labels on form elements to be bold if they're mandatory
<beuno> I somehow feel explaining that won't help
<beuno> but there you have it   :)
<thumper> tal:define="classname python: "" and show_optional or "someclass""
<thumper> no
<thumper> actually that is exactly wrong
<thumper> :(
<wgrant> Sounds like you need Python 2.5.
<beuno> I know I've done this before
<thumper> switch "someclass" and ""
<mwhudson> tal:attributes="class python:[None, 'show_optional'][bool(show_optional)]"
<mwhudson> it's horrific, but it'll work
<thumper> ick
<thumper> that'd need a comment for sure
<beuno> UI is fun
<mars> tbh, I'd bounce it
<mars> put that in the view
<mwhudson> beuno: nicer would be widget/fmt:label-css
<mwhudson> and some magic in tales.py
<beuno> yes, nicer is more expensive than I can afford
<thumper> why?
<beuno> so I think I'm just not doing it
<mwhudson> how is that expensive?
<thumper> beuno: get wgrant to write it :)
<beuno> I need to start poking in more places, test it, etc
 * wgrant fortunately has lectures today.
<mwhudson> beuno: you're not very receptive (and this is a good thing!) to "too much work" when you do ui reviews :)
<beuno> mwhudson, heh
<beuno> which is why I try and do as little code as possible in LP  :)
<beuno> I'm a terrible judge for myself
<beuno> but somebody has to make forms stop sucking
<thumper> beuno: so just DTRT and do it properly :)
<mwhudson> beuno: i'm happy to help with coding suggestions, etc
<mwhudson> but this is not the place for hacks
<beuno> I'm going to wait and see if changing this in forms breaks dozens of tests or not
<beuno> and decide based on that if I don't implement that UI feature at all
<beuno> I won't hack it in, it's not worth it
<beuno> so either DTRT or not doing that change
<thumper> mwhudson: how do I ec2test jml's branches without grabbing and pushing them myself?
<thumper> mwhudson: is the a convenient param?
<mwhudson> thumper: ec2test.py lp:~whatever
<thumper> ah fantastic
<thumper> mwhudson: I'll test submit jml's branches
<mwhudson> thumper: you'll need to supply a submit address somehow if you want to use -s
<mwhudson> and submit branch
<thumper> mwhudson: ok
#launchpad-dev 2009-07-29
<thumper> mwhudson: it seems to me that the submit branch and email should be extracted correctly without any extra setup by me...
<mwhudson> thumper: where from?  they're currently set by locations.conf for local paths
<thumper> nope
<thumper> hmm
<mwhudson> so if the path is some launchpad url...
<mwhudson> can you put a regex in locations.conf?
<thumper> don't know
<mwhudson> you could probably set something up to work magically, but you're unlikely to have it set up now
<thumper> is it  --pqm-public-location that I need
<thumper> I'm finding the help not helpful with this respect
<mwhudson> no, not that
<mwhudson> --pqm-submit-location
<mwhudson> oh, maybe you don't need that actually
<thumper> mwhudson: it says it should default to devel
<mwhudson> thumper: right
<mwhudson> maybe it's just the submit address you need to set?
<thumper> there isn't an option for that :(
<mwhudson> yeah
<mwhudson> i guess you can set it in the global config while you run the command>
<mwhudson> ?
<mwhudson> and file a bug
<thumper> hmm
<thumper> mwhudson: I hacked ec2test :)
<thumper> mwhudson: I should submit the fix
<mwhudson> hehe
<mwhudson> yeah
<thumper> spm: it seemed that the email for devel r8980 didn't get sent out
<thumper> spm: around to help chase?
<spm> thumper: hmmm. not good - I assume you're suggesting the branch scanner? or?
<thumper> spm: no, it *should* be in the logs for the revision mailer
<spm> 'k, looking
<thumper> I can't remember where that job runs
<thumper> probably on the same machine as the scanner
<mwhudson> loganberry i think
<thumper> maybe
<thumper> syncing the logs with devpad would be a start
<thumper> spm: sendbranchmail on loganberry
<thumper> spm: srv/launchpad.net-logs/scripts/loganberry-bzrsyncd is where it goes
<spm> thumper: yeah, that log is not terribly helpful for informing on if a specific XYZ worked tho :-(
<thumper> :(
<thumper> ok, db queries here we come
<spm> is syncing atm...
<spm> fwiw
<mwhudson> look for oopses too?
<spm> gah. going blind. didn't notice the diff between sendbranchmail.log & the dir sendbranchmail
<spm> mwhudson: thumper: https://pastebin.canonical.com/20506/
<thumper> spm: ah
<thumper> spm: this is the fix that abentley has landed
<thumper> spm: we should look to get that cherry picked somehow
<thumper> yay for fixes already landed :)
<spm> heh
 * thumper afk to get Maia
<mwhudson> lunchtime
<thumper> for an api write operation does the changed object get streamed back to the launchpadlib client?
<thumper> also, is the vocabulary checked?
<thumper> hmm...
<wgrant> A Choice argument is checked server-side.
<wgrant> And client-side in really new launchpadlibs.
<wgrant> And IIRC the object does get given back in response to a PATCH, but I'm not sure why you care.
<thumper> wgrant: IBranch.setOwner
<thumper> wgrant: changes the state of the branch
<thumper> wgrant: and possibly the branch name if there is a clash
<wgrant> Oh dear.
<wgrant> That's awkward.
<thumper> wgrant: I'm wondering what gets sent back to the client
<thumper> ideally I'd like it to just work
<wgrant> It should.
<wgrant> Because launchpadlib just looks at self_link to work out the name.
 * thumper looks in registry/interfaces/person.py
<wgrant> I know one good way to find out.
<thumper> wgrant: which is?
 * thumper broke buildbot
<wgrant> thumper: Use launchpadlib on launchpad.dev.
 * thumper broke the build, not buildbot
<wgrant> buildbot doesn't seem to need people to break it.
<thumper> heh
<mwhudson> thumper: a http redirect gets sent back if the url has changed, i think
<thumper> mwhudson: hmm...
<mwhudson> yeah
<thumper> mwhudson: http://pastebin.ubuntu.com/235582/
<thumper> mwhudson: for my testfix :(
<mwhudson> lazr.restful/src/lazr/restful/_resource.py:1099 and tharabouts
<thumper> mwhudson: what is the meaning to the client though?
<mwhudson> thumper: r=me with a side of 'ya doctests'
<mwhudson> thumper: it will go to the new url and presumably get the new version of the object
<mwhudson> i guess
 * thumper hopes
<mwhudson> i dunno, like wgrant says, try it, i guess
<mwhudson> thumper: in some sense, it's not so different to any other modifying operation, is it?
<thumper> mwhudson: well... this one will change the canonical_url of the branch
<mwhudson> thumper: oh right
<thumper> mwhudson: generate_entry_adapter -- hmm, I wonder if lazr is gaining adapters
<mwhudson> there are certainly adapters all over
<mwhudson> thumper: about your testfix, are you sure that that's the only test that is broken?
<thumper> yes
<mwhudson> cool
<thumper> almost
<thumper> pretty sure
<thumper> 99.9%
<thumper> can I pqm-submit a branch on LP without a local copy?
<mwhudson> you can write the submit mail by hand i guess
<mwhudson> bzr pqm-submit --dry-run in some other branch, copy paste into email client, change, send
<mwhudson> spm: you'd think in the 1hr10 since i pinged you, i could have written the queries :)
<mwhudson> but no
<thumper> oh the irony
<thumper> bzr branch lp:hg-git
<thumper> I just approved the import
<spm> mwhudson: heh. "no comment"
<mwhudson> spm: select distinct 'https://code.edge.launchpad.net/' || unique_name from codeimportresult, codeimport, branch where codeimportresult.code_import = codeimport.id and codeimport.branch = branch.id and codeimportresult.log_excerpt like '%extant%';
<mwhudson> spm: staging is fine
<thumper> oh FFS
<spm> mwhudson: timed? or results?
<mwhudson> spm: results
<thumper> ok
<thumper> got jml's branch in pqm
<thumper> spm: do you know if our buildbot is smart enough to check the commit log before saying testfix mode?
<thumper> spm: although I have a feeling our commit regex won't allow [testfix] through until we are in testfix right?
<spm> thumper: off hand I don't know - I suspect it's a mix of pqm'isms vs buildbot'isms working in new and interesting ways.
 * thumper waits until buildbot complains before submitting the testfix
<spm> thumper: the not-thinking-hard-eyeballing suggests that being in testfix won't matter to the RE
<spm> as in both the success and failure configs look for testfix
<thumper> really?
<thumper> spm: so I could submit a [testfix] now?
<thumper> spm: does [testfix][r=xxx] with no [ui work?
<spm> thumper: to devel? no. db-devel - I believe! will
<thumper> gah
 * thumper is shepharding his 7th landing
 * thumper waits for buildbot to crap itself to submit testfix...
<thumper> spm: what is the regex for devel in failure mode?
<spm> thumper: commit_re=(?is)^\s*\[testfix\]\s*\[(?:release-critical|rs?=[^\]]+)\]
<thumper> spm: ta
<thumper> mwhudson: a start -- https://dev.launchpad.net/Code/PostThreeDotO
<wgrant> Mmmm. Sphinx.
<thumper> maybe...
<wgrant> Only maybe? Sad.
<thumper> we need to get the entire idea approved first
<thumper> but I'd like to see it happen
<thumper> I think it has a good chance as far as any site hosting goes
<thumper> my thoughts are that we'd have an option of "No processing" or "Sphinx it" on the contents of a branch
<thumper> to host the content
<thumper> although we should also support existing documentation in a subdirectory of an existing branch
<thumper> hence the other task
<wgrant> Right.
<thumper> that was basicly a brain dump from Monday
<thumper> I've been meaning to write it up
<thumper> I'm sure it isn't complete
<thumper> as I said, doesn't count any bug work
<thumper> nor general UI improvements
<thumper> and some tasks are way too big
<mwhudson> thumper: do you think 'crack of the day' ppas need any work from our side?
<thumper> mwhudson: possibly
<thumper> I'd need more understanding of how it'll hang together
<rockstar> Ah man, who broke the build?
<thumper> me
<thumper> testfix submitted already
<rockstar> You da man.
<rockstar> I always panic just a bit when I end up on the buildbot blamelist
<rockstar> I can never remember where the "Subscribe" link on wiki pages is.
<wgrant> I looked this morning and couldn't find it.
<wgrant> Is it actually there?
<wgrant> The source suggests not.
<rockstar> wgrant, yeah, I can't find it, but I'm sure I'm subscribed to some pages.
<wgrant> You can always do it manually from UserPreferences.
<lifeless> is lp still hot?
<wgrant> It was 24 hours ago.
<wgrant> It is.
<thumper> lifeless: why?
<lifeless> thumper: I'm about to setup a test environment for faster commit with iter changes, and lp is a half decent test case
<lifeless> but I want the code today
<thumper> lifeless: herb has a tar ball
<thumper> lifeless: somewhere
<lifeless> its ok
<wgrant> http://people.canonical.com/~herb/
<lifeless> too late:P
<lifeless> my link is already streaming down bzr+ssh goodness, saturated link, kgo
<wgrant> Ah.
<wgrant> Is LP still hot just because nobody has cooled it yet, or is there still load?
<lifeless> I don't believe we even tested bzr+ssh scaling
<lifeless> while bzr has http bugs the request rate is hundreds, or even thousands of times higher than it should be
<lifeless> so load can be extremely deceptive here
<wgrant> HTTP 2a does seem pretty sucky.
<wgrant> But a bzr+ssh push of no new revisions seems to eat about 1MB too.
<wgrant> (of LP, that is)
<lifeless> probably tags
<lifeless> maybe a regression causing it to send the adjacent (thus tip) inventory again as well
<lifeless> how much was the db-devel branch? 130MB or something?
<wgrant> Disk- or transfer-wise?
<lifeless> disk
<lifeless> 53M transferred so far
<wgrant> 220ish MB, I think.
<wgrant> The wiki says 150MB, but I've never seen it that small.
<lifeless> probably nearly 30% overhead :P
<wgrant> Overhead on top of what?
<lifeless> its what happens when you request 500 byte windows :P
<lifeless> vs bzr+ssh
<wgrant> Ah.
<wgrant> it was 280MB over HTTP.
<wgrant> Ended up as a bit more than 220MB on disk.
<lifeless> ah
<wgrant> My local non-shared branch is being very slow, but just crossed 210MB.
<wgrant> 237MB.
<lifeless> 90M
<wgrant> I hope that's not finished.
<lifeless> no, not yet
<lifeless> but soon
<wgrant> Odd definition of 'soon'.
<wgrant> Or you have too much faith in 2a.
<lifeless> 2a is seriously good
<lifeless> http bugs aside
<wgrant> It is.
<wgrant> I'm really pleasantly surprised at how well it works with LP.
<wgrant> It's actually pretty fast, apart for no-op pushes.
<spiv> lifeless: feel free to update the size estimate on the wiki... I added the 150M based on what an lp dev said on this channel, but I guess they were wrong...
<lifeless> wgrant seems to have concrete figures :P
<wgrant> spiv: It was updated on release night with my 280MB figure, in addition to the 150.
<lifeless> spiv: I'm getting choppy performance
 * wgrant disappears.
<lifeless> dropping down to 60K and back up, sometimes even lower
<lifeless> we should debug and fix
<spiv> wgrant: oh, right.  Thanks.
<spiv> lifeless: yeah.
<spiv> lifeless: if you have a hpss log of it maybe dump it in a bug when it's done?
<lifeless> I don't
<lifeless> its running on my games machine; my laptop is too small to benchmark (disk space, not cpu/memory)
<lifeless> mwhudson: so while you look at cscvs
<lifeless> I asked a question on launchpad-code a few days back
<lifeless> about an import that is being killed, but seems to be not a cscvs bug when I downloaded the [massive] log
<lifeless> for gnutella
<lifeless> sorry, not gnutella, but limewire
<mwhudson> oh yes, the cvs logs are ridiculously enormous :(
<lifeless> anyhow
<lifeless> is there some way to say 'really, just let this one go'
<mwhudson> lifeless: it's failing with memoryerror?
<lifeless> I don't remember the problem offhand
<lifeless> I put it all in the question
<mwhudson> it looks like it
<lifeless> oh yeah
<lifeless> so
<lifeless> I'm not sure what to do about it
<mwhudson> lifeless: i don't understand your question, it's clearly a memory error?
<lifeless> wgrant: completed
<lifeless> mwhudson: is there anything that can be done?
<mwhudson> lifeless: i guess we could do the import on some beefier machine
<mwhudson> updates presumably won't be so crazy
<lifeless> It should be a one time pain
<mwhudson> i don't suppose we have big-ram machines just lying around the dc unused though
<lifeless> wgrant: 33 minutes to download, 176MB on disk.
<lifeless> mwhudson: how big are the machines the imports run on?
<lifeless> are they ulimited?
<mwhudson> no ulimit
<mwhudson> i think they have 2 gigs
<lifeless> whee
<lifeless> hungry hungry hippo
<jtv> hi folks
<mwhudson> uh, strange, they all report different 'total' amounts in free: 4152600, 4087172, 4927824
<mwhudson> lifeless: i see the last failure was a while, we could try again, maybe the machines got upgraded
<mwhudson> (not sure though)
<lifeless> please
<lifeless> thats a fairly cheap experiment
<lifeless> also, bzr+ssh, 6 times faster than http. FTR.
<lifeless> maybe more
<mwhudson> oh good, it started on nemayer
<mwhudson> not galapagos which has a 1.1 gig bzr-git import going
<jtv> I've been unlucky with my connections to the DC all day... is everyone downloading the launchpad source or is it just me?  (I tried different providers, different hardware, different DSL lines)
<lifeless> heh
<mwhudson> https://code.edge.launchpad.net/~vcs-imports/9p-linux/trunk -> 2009-07-29 05:44:31 INFO    fetching revisions 35908/155568
<lifeless> jtv: its just you
<jtv> of course.  evil eye.  :(
<lifeless> jtv: even at peak lp code downloads would likely have not shown up :)
<mwhudson> that's a lot of revisions
<lifeless> mwhudson: and a metric fuckload of branches
<jtv> my ping is well <.5 seconds, packet loss <5%, but bzr seems really picky
<lifeless> jtv: 5% is huge packet loss
<lifeless> tcp goes shite before that
<jtv> lifeless: not in the rain season crossing both major oceans :)
<lifeless> that said, if you're pulling from lp:launchpad, you'll be doing http, and that will fail hugely.
<lifeless> because bzr makes thousands of requests at the moment, its fairly certain you'll get failures to connect and things like that.
<jtv> It's going over http now?
<lifeless> if you use an lp: url for launchpad yes
<lifeless> its a temporary measure, but it has the effect, due to bzr bugs (and http not being as efficient as bzr+ssh) of being slower and more fragile
<jtv> hmm... that could have interesting consequences with the whole censorship infrastructure inbetween.
<lifeless> give it explicit bzr+ssh urls
<mwhudson> lifeless: the limewire import just hit a gig resident
<lifeless> woo
<lifeless> 1.3 seconds to 0.3 seconds with this patch
<lifeless> for commit sourcecode/Makefile
<wgrant> lifeless: Huh. I wonder if mine needs to be repacked.
<wgrant> It was an HTTP checkout, I guess.
<wgrant> Oh dear, I'm in the blame list.
<lifeless> its all your fault
<wgrant> I do that.
<noodles775> Morning ppl
<wgrant> Morning noodles775.
<noodles775> Afternoon wgrant.
<noodles775> wgrant: how are things going? have you found any specific features you're keen to work on (or ideas of your own?) It's been great seeing the contributions you've been making already!
<wgrant> noodles775: If development wasn't going in the direction it is now, I'd fix some UI stuff, but there's probably not much point now.
<noodles775> wgrant: what do you mean? which direction is it going that you don't like? Or did I misunderstand?
<wgrant> noodles775: Lots of UI changes going on, I believe.
<wgrant> Not that I don't like it.
<noodles775> wgrant: yes, but why does that stop you from fixing UI stuff (only if you wanted to of course)
<noodles775> ah, you mean small ui bugs that will be fixed or cease to exist as part of larger UI changes?
<wgrant> noodles775: Exactly.
<wgrant> They'll presumably be noticed and resolved as people go over pages.
<noodles775> wgrant: well, if you're keen (and only if you want to), you can join in the larger UI refactorings too...
<wgrant> noodles775: I may be tempted once I see what is actually happening.
<noodles775> I'm trying to ensure that the things I'll be working on will be all openly planned etc., then divided up into small bugs (associated with a blueprint) etc. etc.
<wgrant> As it's not at all clear at the moment.
<wgrant> How many pages are actually being redesigned rather than just transitioned to the new base layouts which seem to be the same?
<noodles775> wgrant: OK, so some pages (pages which don't have portlets on the side etc.) can just be updated to the new templates and not much will change (other than the navigation etc. which is also not finalised)
<wgrant> But other pages (eg. the project page) seem to be getting completely redone.
<noodles775> But many will needto be re-designed to match the new 3-0 layout - the biggest factor that I see there is that portlets/info in the side bar are restricted to a defined set of info (actions etc.)
<noodles775> exactly.
<wgrant> I'm glad that use of portlets is becoming well-defined.
<wgrant> As it's all a bit stupid at the moment.
<noodles775> wgrant: within soyuz, the main pages we've identified that will need to change are the archive index (well, ppa index), as you should have seen from my previous email...
<noodles775> Yes, it'll be good for it to be consistent rather than just "oh this will fit here".
<wgrant> Well, on some pages (eg IBuild:+index), I have to wonder how anybody ever thought that.
<wgrant> But I suppose that page has evolved.
<wgrant> And slowly grown all of the information in the portlet.
<noodles775> Yep... often the reason ... exactly. and the reason gets lost... which is why I'm trying to start some permanent use-case information on each page.
<wgrant> A good idea.
<noodles775> So as we go to work on a bug or feature that involves the UI, we have a standard reference (that can of course be updated).
<noodles775> So for example, https://dev.launchpad.net/SoyuzBuildersIndexPage
<wgrant> Particularly if that information is public, so the users can make their use cases known.
<noodles775> is very empty, because I don't yet the real use-cases for this page, and so I couldn't go much further with a mock design.
<noodles775> Exactly.
<noodles775> I'm also drafting: https://dev.launchpad.net/SoyuzDistroSeriesQueuePage
<noodles775> but again, there are a lot of unanswered questions for me for the queue page... I'll be trying to get some input from people who use the queue later today so we can start doing some mocks there too.
<wgrant> The queue page is a lot less painful than the console scripts, as the console scripts have to initZopeless every time, which takes forever :(
<noodles775> wgrant: except the UI for the queue doesn't get used much as it's also broken from the archive admins p.o.v.
<noodles775> (I'll paste in the info from historical emails into that page...)
<wgrant> noodles775: Right.
 * mwhudson no longer here
 * maxb is a bit surprised that the contributor agreement process doesn't ask you to specify your LP id when filing your agreement
<wgrant> Has the Continue button in the popup help gone bad recently for anybody else?
<wgrant> It is missing its text in at least two browsers I've tried.
<wgrant> (the HTML seems clearly wrong)
 * noodles775 tries the help on ppa page...
<noodles775> wgrant: yep, looks like it has... worth a bug to lp-foundations? (I think?)
<wgrant> And a patch when I get home, methinks.
<noodles775> :)
<noodles775> wgrant: OK, I've added https://dev.launchpad.net/SoyuzDistroSeriesQueuePage/Inputs
<noodles775> (with a link from the parent page), if/when you're interested.
<al-maisan> Good morning!
<noodles775> G'day al-maisan :)
<al-maisan> Tach noodles775 :)
<adeuring> good morning
<al-maisan> moin adeuring
<adeuring> hi al-maisan!
<mrevell> Hi!
<bigjools> morning
<mrevell> yo dude
<al-maisan> mrevell and bigjools: good morning
<mrevell> guten morgen
<wgrant> Evening.
<wgrant> bigjools: Thanks.
<bigjools> wgrant: okay!  What for? :)
<wgrant> bigjools: Landing my branch.
<bigjools> ah, you're on the ball
<wgrant> Well, there was an email from Launchpad sitting in my inbox.
<wgrant> Hm, that's a bit unfortunate.
<wgrant> All these devel commits show up without the MP metadata.
<wgrant> As the merges were proposed to db-devel.
<bigjools> wgrant: fancy looking at a mockup of the new DSP page?
<wgrant> bigjools: Sure.
<bigjools> http://people.canonical.com/~ed/dsp_mockup.png
<bigjools> ignore the crap sample data!
<bigjools> I gimped most of it
<wgrant> bigjools: Where is that expander expanded from?
<bigjools> it's  bodged from the same XHR done with the PPA rows
<wgrant> bigjools: I mean, what's it doing there?
<bigjools> hang on, phone
<deryck> Morning, all.
<bigjools> wgrant: ok I am going to tweak the image a bit, but basically the expander will open for each package version on the page, giving you more detail about it and linking to other pages.
<wgrant> bigjools: Each version being those in the pocket listing?
<wgrant> That sounds good.
<wgrant> Particularly if that also applies to +publishinghistory.
<bigjools> yep
<bigjools> wgrant: well that's a separate page, I'll come to that later
<bigjools> wgrant: but I wanted your feedback on the basic direction of the design, and whether you think it needs more/less data.
<wgrant> bigjools: I'm not terribly pleased about the removal of the almost-changelog, but that is probably more effectively replaced with a batched +publishinghistory.
<wgrant> And it's not too bad, since the latest upload in each pocket is just a click away.
<wgrant> Upstream associations would be nice, but they suck at the moment so it's no loss.
<bigjools> wgrant: you'll get the "almost" changelog in the expander sections, just not for all versions any more, which we want to move to +publishinghistory
<bigjools> the description will be the upstream's BTW
<bigjools> including the logo
<wgrant> bigjools: Ah, useful.
<bigjools> to fix https://bugs.edge.launchpad.net/soyuz/+bug/73116
<mup> Bug #73116: Source package pages don't describe what the package is for <pkg-nav-redesign> <ui> <Soyuz:Triaged> <https://launchpad.net/bugs/73116>
<wgrant> Yep.
<wgrant> I wondered how you were going to do that.
<bigjools> it depends on the packaging linkage of course
<wgrant> Yep.
<wgrant> I think initialiseFromParent needs to be taught to copy those.
<wgrant> Or it's going to be pretty bad.
<bigjools> interesting idea
<bigjools> file a bug about it and start some conversations
<wgrant> It's a bit silly to have to redo them for every series.
<bigjools> agree
<wgrant> Although it'd be even better if the productreleasefinder would work for more products, and compare against .orig.tar.gzs in the distro.
<wgrant> bigjools: Is that a Registry or Soyuz bug?
<bigjools> wgrant: registry
<bigjools> wgrant: although file on launchpad itself and matsubara or Ursinha will triage it
<bigjools> wgrant: reload that mockup
<wgrant> bigjools: You need a link to the SPR as well as to expand the expanders. You also need a link to the product, and to move the publishing history link above the PPAs.
<bigjools> wgrant: heh, you can argue with beuno about the latter
<wgrant> bigjools: It's only one line now.
<wgrant> But I shall prepare for an epic battle.
<bigjools> remember that the PPA section is unexpanded by default
<wgrant> True.
<wgrant> But still.
<gary_poster> maxb: looks like https://edge.launchpad.net/~gary/launchpad/zbuildout already is subscribed to by Launchpad Hackers, which is what I would have expected to be necessary.  Can you access it now?  Am I right in assuming that you are a member of Launchpad Hackers?
<maxb> I can't access it. I'm a member of ~launchpad-dev
<noodles775> Wow... who can I thank for the ec2test speedup?
<noodles775> Total: 23717 tests, 0 failures, 0 errors in 160 minutes 21.996 seconds.
<salgado> noodles775, jml, I think. he landed a change to disable staggered retries on pagetests
<noodles775> Sheesh... that's a huge time difference! Well done jml :)
<jml> well, I also changed the default instance type to one that was twice as fast
 * jml isn't actually here
<noodles775> ah, cprov just mentioned that might have happened. Thanks jml - and enjoy your evening!
<mars> x1.large?
<mars> nm, I'll just check the source
<cprov> c1.xlarge
<bac> noodles775: my last run of ec2test was on thursday and it took 158 minutes using c1.xlarge.  so it looks like the tests didn't speed up at all it's just the larger instance by default.
<noodles775> bac: yes, seems so (I hadn't used the large instance before)
<bigjools> tests are faster for me locally
<bigjools> 185m -> 160m
<bac> bigjools: really?  since when?  was it jml's change?
<bac> wow, that's great.
<bigjools> could be!
<bigjools> or a new storm?
<mars> I think you are supposed to conceive of a hypothesis before the experimental speedup, not the other way around...
<bac> mars: but it's not our experiment.  we're just bystanders trying to figure out what happened.  :)
<maxb> gary_poster: If you go to https://code.edge.launchpad.net/~gary/launchpad/zbuildout/+edit do you have a "Keep branch confidential" checkbox you can untick ?
<gary_poster> maxb: yes, which I unticked immediately after subscribing launchpad-dev, so now you are doubly permitted. ;-)
<maxb> aha
<gary_poster> maxb: last I checked, the tests were not starting, so I'll have some work on this to get it back up.  I suspect I won't be back on this till tomorrow at the earliest.  That said, don't worry, there's plenty of internal pressure on this being done too. ;-)
<maxb> Is having ~launchpad-dev subscribed going to send push notifications to the mailing list?
<maxb> gary_poster: No problem, I'm attacking the issue from the other end anyhow. I have a LP running on Python 2.5 that works, and a quite a lot of the tests pass straight away
<gary_poster> maxb: push notifications: good question, probably.  Python2.5: good.  I hope you have been coordinating with barry, then?
<maxb> push notifications: ok, I'll unsubscribe the team then. coordinating: no. I've been loitering here and mentioning it but no one has mentioned anyone I might want to talk to other than yourself
<maxb> I've documented what I've done on http://dev.launchpad.net/LaunchpadOnKarmic
<gary_poster> maxb: ah ok.  yeah, barry had a branch and was doing what you are doing.  He and I both have been busy with some other tasks; he is off today, I think.
 * maxb sets /notify barry :-)
<mars> maxb, you may want to start a thread on the dev list, let everyone know about your instructions, and cc barry on it
<gary_poster> maxb: wiki page: awesome!  mm...would you be willing to send an email to launchpad-dev mentioning your work and its existence, and the fact that you are trying to coordinate with me and Barry Warsaw, just so more people know of your efforts?  I think the team ought to know about this, and that's also another way of pinging Barry.
<gary_poster> The other person who will want to know/coordinate is my boss, Francis Lacoste (probably flacoste here) who is not available till late next week.
<mars> yeah, what gary said :)
<maxb> sure - I'll email launchpad-dev@ this evening then
<mars> gary_poster, awesome, debian has a command for proxying stuff into xvfb: xvfb-run: http://www-linux.gsi.de/cgi-bin/man2html?xvfb-run+1
<mars> that means I can do this in ec2test-remote.py:
<mars> xvfb-run -s "screen 1024x768" make jscheck
<mars> no messing around with hand-hacked init files, or with starting and stopping xvfb in the runner
<gary_poster> mars: wow!  so...that means that your job suddenly becomes a lot easier?
<mars> oh yes :)
<gary_poster> awesome!
<mars> gary_poster, is it easy to add a new buildbot stream?
<mars> or slave?
<gary_poster> mars: no, not at the moment.  It requires a new image ATM.
<mars> a new buildbot image?
<gary_poster> mars: a new slave image, yes
<mars> ok
<mars> then I should look at landing the image builder script
<mars> and adding a slave generator to it
<gary_poster> mars: but you know, effort is relative.  but adding a slave is something that francis usually schedules, rather than being a "hey would you please..." kind of task.
<mars> heh
<mars> what is the most difficult part of setting it up?
<gary_poster> mars: slave generator would be cool.  I also want to change buildbot to be able to use latent buildslave images that are generic.  To do that requires changes to buildbot master and slave code, using the twisted perspective broker stuff, so it would be cool but potentially time consuming to figure out.
<gary_poster> that would mean that you wouldn't need new images for new slaves
<gary_poster> just for updates
<gary_poster> and you would only need one slave image for the update
<gary_poster> instead of one per slave, as it is now
<mars> gary_poster, could the ec2 slave code go into an s3 bucket, and be run from there?
<mars> nm, guess not
<mars> you are right
<mars> you need to have a custom configuration somewhere that tells each slave instance "You are testing foo.  Here's how"
<mars> and that configuration needs to stick between slave restarts
<gary_poster> mars: most difficult: probably the slave image.  Other than that, it's just a slow process to test, and restarting the master is a pain because it throws our whole test/merge process into a dither.
<mars> ah
<gary_poster> mars: the description of how to test is already contained in the master, not the slave
<gary_poster> mars: this is a "simple" problem of authentication
<gary_poster> mars: usually (non-latent slave) a slave says to the master, "hi, I'm the slave named foo and my password is bar.  What would you like me to do?"
<gary_poster> mars: the latent slave story needs to be able to say "hi, slave.  You are named foo.  I don't care about a password.  Now here's what to do."  or similar.
<mars> ah, ok
<gary_poster> It's a further refinement of the work I already did.  It wasn't necessary for buildbot to launch last year, so we postponed.
<mars> pragmatism :)
<gary_poster> yup :-)
<bac> reviewers meeting now in #launchpad-meeting.
<mars> bigjools, IIRC, barry and I talked about making our code standards a bit more public: the Launchpad Code Guidelines, or LCG for short
<bigjools> yes, we should definitely do that
<mars> because they are well-developed, go a bit beyond PEP008
<mars> and they cover js, and component design
<bac> hi kfogel
<noodles775> sinzui: sorry if you already responded (my connection dropped), but is there any news on when we can start landing simple templates that update to the 3-0 base layout?
<sinzui> noodles775: most the the template/macro/css is in trunk now
<sinzui> noodles775: There is a small css issue that breaks the side portlets in main_side. I have it fixed in a branch I am working on
<noodles775> sinzui: great! Though the other day you mentioned structured headings being the problem. Is that sorted now?
<sinzui> noodles775: I can merge my fix into the edit work that beuno was looking at. I hope to land it today.
<sinzui> the heading is in trunk...
<sinzui> The heading is incompatible with the h1 edit tite widget
 * sinzui has a fix for that too
<mars> gary_poster, can ec2test.py use an already-running instance?
<beuno> sinzui, https://code.edge.launchpad.net/~beuno/launchpad/finally-fix-forms/+merge/9427
<beuno> sinzui, all yours
<gary_poster> mars: no
<mars> could I hack it so it does? :)
<sinzui> beuno: fab
<gary_poster> mars: ...yes, probably, assuming that the instance has already been set up the way that the script expects.
<sinzui> beuno: Should I send my edit page changes for review today? Are there any issues you want me to fix with annoucement and project modification pages?
<beuno> sinzui, let me take a quick look over them again knowing that the forms are now fixed
<gary_poster> bigjools: do you happen to remember that issue we had with a function that returned a security-proxy wrapped object, and we felt it was unnecessary, and we got it changed?  If so, could you job my memory as to what the function did?  I'm trying to write up how we use security, per ye olde reviewers meeting action item of mine.
<bigjools> one sec, phone
<gary_poster> bigjools: np, thanks
<mrkid> can somebody help me?
<mrkid> i wanna setup launchpad on my hardy server
<bigjools> gary_poster: it was something that returned the view in a menu class
<gary_poster> bigjools: ah right, thanks, perfect
<bigjools> views are not proxied anywhere else
<gary_poster> right cool
<mars> mrkid, well, we can help you set up a development instance.  have you checked the instructions on http://dev.launchpad.net/Getting ?
<bigjools> gary_poster: welcome!
<mars> :/
<mrkid> mars: yes, but there are some errors
<mrkid> i try : bzr --no-plugins cat http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/utilities/rocketfuel-setup > rocketfuel-setup
<mrkid> but i got: bzr: ERROR: Unknown branch format: 'Bazaar Branch Format 7 (needs bzr 1.6)\n'
<mrkid> how can i fix it
<mars> are you using at least bzr 1.16.1?
<rockstar> mrkid, you'll need to upgrade bzr
<mrkid> thanks all, i will try
<mars> mrkid, the instructions are on the /Getting page, in the "Overview" section
<mrkid> thanks
<mrkid> i got it
<sinzui> beuno: join #launchpad-reviews
<mrkid> #launchpad-reviews
<sinzui> bac: ready for our call
<bac> sinzui: yes
<NCommander> wgrant, ping?
<kfogel> bac: hey
<noodles775> Night all.
<sinzui> beuno: cprov help. I do not know how to get the version and date for the Packages in distribution portlet in this design: https://devpad.canonical.com/~curtis/LP_projectdetail.png
<cprov> sinzui: let me check.
<sinzui> beuno: cprov: I really do not know what kind of object I am looking at this this design
<cprov> sinzui: I think you can follow the Packaging table to get the corresponding SPN.
 * sinzui looks
<cprov> sinzui: actually you find the ancestry SPs (sourcepackagename + series) in the Packaging table, then you can expand it up (good it doesn't make any sense explained that way)
<mrevell> Okay guys, speak to you tomorrow :)
<sinzui> cprov: can you provide a url or a template name? I am truly lost
<bigjools> sinzui: look at the existing distribution source package page
<bigjools> it has packaging stuff at the top
<cprov> sinzui: okay, let's say the bazzar upstream product was package in dapper, edgy and feisty as 'bazzar' and since gutsy it was packaged as 'bzr' ... You will have 2 entries in Packaging for that product [(bazzar-SPR, dapper), (bzr-SPN, gutsy)]
<bigjools> distributionsourcepackage-index
<sinzui> okay, that helps. The code looks like I was getting a SourcePackage...no distro
<cprov> sinzui: SP == (SPN, DistroSeries)
<cprov> sinzui: none of the widgets we have atm does the right thing with Packaging records, i.e. it's not expanded properly.
<cprov> sinzui: see `Product.sourcepackages`
<sinzui> well, package/version is gives me nothing. Either sampledata is invalid, or I do not have the right objects
<sinzui> cprov: that is indeed what I am iterating over, but the version is always nothing
<cprov> sinzui: most likely sampledata is crap
<sinzui> \o/
<cprov> sinzui: but even if it was good the result won't be expanded to newer series as we want it to be.
 * sinzui looks for something other than sample firefox
<cprov> sinzui: https://launchpad.dev/netapplet is satisfactory
<sinzui> cprov: thanks. I am still not getting a version from the Product.sourcepackages, but I at least know I should be able to do it
<cprov> sinzui: SPs do not have a version, one sec
<sinzui> cprov: okay, this is what I was thinking...we want to show something *different* on the page, or the designer did not know that SP and DSP are not the same.
<cprov> sinzui: SP.distinctreleases ... have
<sinzui> plural? Again this lead me back to the moment I asked for help. This design is flawed, it does not represent how things work in launchpad. cprov looking at that picture, does the packaging make sense?
<cprov> sinzui: in some extent SP & DSP are pretty much the same, they are just bound to different contexts (distroseries vs. distribution). They both do not have version, but instead 'versions'
<cprov> sinzui: it does, you will have multiple versions under 'gnome-panel in Ubuntu Gutsy'
<cprov> sinzui: or better, if you want to be accurate with the mockup, use the latest version
<sinzui> cprov: currentversion?
<cprov> sinzui: it's not implemented by SP
<cprov> sinzui: SP and DSP both implement 'currentreleases' but it also return multiple releases corresponding to each pocket.
<sinzui> yuck. This design is flawed. I will ignore this line until beuno can specificy what he wants.
<bac> kfogel: hi, again
<cprov> sinzui: okay
<kfogel> bac: hello
<bigjools> gary_poster: do you know what "belt and suspenders" means in British English? :)
<rockstar> bigjools, I'm curious as to what it means, especially since it's you pointing it out.
<bigjools> rockstar: suspenders here are what women use to hold up stockings
<bigjools> belt & suspenders has entirely different connotations
<rockstar> bigjools, ah, we call those garter belts.
<bigjools> braces == suspenders, for future reference :)
<gary_poster> bigjools: oh. :-)  oops.
<bigjools> just don't use the word "fanny" here ok?
<gary_poster> bigjools: lol, ok
<bigjools> :)
 * rockstar goes to eat a foods.
<sinzui> cprov: sp.distinctlreases is indeed what we want, it just does not work as the designer intended. I will use it and explain why.
<cprov> sinzui: okay. Are we continuing with the unexpanded-packaging issue ?
<sinzui> cprov: The design suggests that I use a dt, dt pair. Since I can have multiple dd per dt, I just iterated over them. I am not showing the pocket information
<cprov> sinzui: I mean 'expanded' in the model domain, not the UI, sorry. I was referring to the `Product.sourcepackages` issue.
<cprov> sinzui: it's unrelated to the UI, redesign, so ignore me for now we can talk about it later.
<sinzui> I think I will solve that another day, after speaking to distro people about what they expect. For now I want to get the UI complete
 * cprov nods
<mwhudson> BjornT: congrats on r9000 :)
<beuno> thumper, are we still on in 30 min?
<beuno> wooo, deryck really nailed description editing
<thumper> beuno: yes
<kiko> beuno, really?!
<beuno> kiko, yes
<beuno> I have an amazing branch here
<beuno> there's a few small UI tweaks to be done
<beuno> but it's minor
<beuno> he claims the code needs cleaning up as well
<beuno> but it works as mocked up
<beuno> the UI is not even interaction, he nailed those
<beuno> it's layout
<mwhudson> morning
<thumper> mwhudson: morning
<thumper> mwhudson: https://code.edge.launchpad.net/~vcs-imports/xizero/master
<mwhudson> thumper: i saw, briefy
<thumper> mwhudson: the git import had an invalid sha1
<mwhudson> thumper: guess a bzr-git bug
<thumper> mwhudson: I'm wondering when we rolled out the fix to cache the sha1 map
<thumper> rockstar: can I line you up for a call after the standup?
<mwhudson> oh right, it failed a while a go
<rockstar> thumper, yessir!
<thumper> rockstar: ta
<mwhudson> thumper: i trashed the copy of the import
<thumper> mwhudson: ok, lets see if that fixes it
<mwhudson> yeah
<wgrant> NCommander: Hi.
<rockstar> mars, ping
<mars> hi rockstar
<rockstar> mars, so BjornT has introduced LP.client.ErrorHandler.  Have you seen it?
<mars> rockstar, nope - have a screenshot?
<mars> or some way to play with it?
<rockstar> mars, it doesn't really have any UI for it.  I've just barely started using it.
<rockstar> mars, although Bugs has been using it.  deryck pointed it out to me.
<mars> what does it do?
<abentley> thumper: skype?
<rockstar> mars, so you create it, define some methods to tell it how to deal with an error, and then on failure of Y.io calls, have it call the error handler.
<thumper> abentley: on with beuno right now
<rockstar> mars, basically, I think we need to unify on the error handling story, because we all seem to be doing different things.
<rockstar> mars, we may need to consult the great and powerful beuno.
<mars> agreed - I'm glad to see someone went ahead and wrote something
<mars> do you know what bugs have the error handling routines do?
<rockstar> mars, they are using overlays to display the error itself, which I partially disagree with.
<rockstar> mars, but I've just been calling alert() with a generic message, and then Y.log-ing the error itself.
<mars> I don't really have an idea for either
<mars> save for turning the broken component red, with a link to the word "Oops!" on it, that pops up the error window :)
<mars> think of it as a non-interrupting error
<mars> otherwise, if anyone has seen how other sites handle AJAX errors, I'd like to hear about it
<rockstar> mars, so, Facebook's (which I see WAY too often) puts up a bit too technical of a message, but having a little overlay telling me something went wrong is nice.
<rockstar> mars, but overlays are a bit intrusive.  I'll raise it for the next UI meeting.
<mars> rockstar, here's a test for what Y! does, at least at the business logic level
<mars> visit http://m.www.yahoo.com/
<mars> click the first left item, "View Yahoo! Sites"
<mars> And then click on the blue + beside "Addresses"
<mars> and you get an overlay
<mars> which is part of their global page error handling structure
<mars> the div is already present in the DOM
<rockstar> mars, yeah, that's pretty good.  I'll bring that up in the next UI meeting, and maybe we'll incorporate that in the 3.0 stuff.
<beuno> sinzui, I've pushed the bold label change, would you approve the branch on the MP?  https://code.edge.launchpad.net/~beuno/launchpad/finally-fix-forms/+merge/9427
<sinzui> beuno: it is approved
<beuno> sinzui, danke
<Ursinha> mwhudson, hiz
<Ursinha> er, hi
<mwhudson> Ursinha: hello
<mwhudson> Ursinha: what have i broken now? :)
<Ursinha> mwhudson, lol
<Ursinha> mwhudson, are you aware of https://bugs.edge.launchpad.net/launchpad-code/+bug/406410
<mup> Bug #406410: Trying to look to the merge proposals details makes edge and staging platforms of lp to timeout <merge-proposal-review> <timeout> <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/406410>
<thumper> who is going to fix the testfix?
<mwhudson> Ursinha: i was not, but i bet i know what the problem is
<Ursinha> mwhudson, better than being aware and not knowing what the problem is :)
<Ursinha> mwhudson, what's that?
<mwhudson> Ursinha: it's a huuuge diff and streaming it from the librarian takes ages
<mwhudson> Ursinha: you can see that the sql time is pretty low
<Ursinha> mwhudson, oh. I see
<Ursinha> mwhudson, indeed, I saw that
<mwhudson> there's a bug about this, which i am currently entirely failing to find
<rockstar> thumper, http://www.fsf.org/blogs/community/savannah
 * mwhudson slaps launchpad
<mwhudson> get faster!
<wgrant> I noticed that launchpad.dev is nice and fast.
<Ursinha> lol
<Ursinha> mwhudson, searching for it
<mwhudson> Ursinha: https://bugs.edge.launchpad.net/launchpad-code/+bug/401554
<mup> Bug #401554: branch merge proposal pages should limit how much diff they show <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/401554>
<mwhudson> wgrant: yes, <1ms network latency and a small database sure help
<wgrant> mwhudson: Yep.
<mwhudson> Ursinha: do you want to make the new bug as a duplicate or shall i?
<Ursinha> mwhudson, I'm doing that now
<mwhudson> cool
<Ursinha> mwhudson, thanks for finding that so quickly
<mwhudson> Ursinha: np
<mwhudson> Ursinha: i guess we should milestone it
<mwhudson> thumper: what do you think to fixing https://bugs.edge.launchpad.net/launchpad-code/+bug/401554 soon?
<mup> Bug #401554: branch merge proposal pages should limit how much diff they show <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/401554>
<Ursinha> mwhudson, that would be great
<thumper> yeah, we should fix that
<thumper> mwhudson: if you have it open, target for 2.2.8
<mwhudson> thumper: done
<thumper> ta
<Ursinha> thanks guys
<mwhudson> np
 * rockstar walks the dog
<mwhudson> i wonder why we're seeing this problem so much more now
<mwhudson> it's been around as long as review diffs
<mwhudson> i guess people must actually be using our software more!
<thumper> beuno: still around?
#launchpad-dev 2009-07-30
<thumper> is anyone fixing buildbot?
<mwhudson> i'm not
<mwhudson> so i guess that means noone is
<mwhudson> thumper: i _presume_ the test missed being updated?
<mwhudson> thumper: hard to see the change on the blamelist that could have caused this
<thumper> mwhudson: yeah
<thumper> mwhudson: I think I've seen this failure before
<thumper> mwhudson: and I'm wondering if it is intermittant
<mwhudson> thumper: does it fail locally?
<mwhudson> gradually getting to the point of being able to test that
<thumper> I'm pulling trunk now
<mwhudson> thumper: it's a really weird failure
<thumper> succeeds locally
<mwhudson> i wonder if we're managing to generate different emails with the same message ids or something
<mwhudson> (which would be bad!)
<mwhudson> i think we must be
<thumper> mwhudson: I bet it is timing based
<thumper> there are 10 recipients and 9 message ids
<mwhudson> thumper: i think i'm right
<mwhudson> >>> from email.Utils import make_msgid
<mwhudson> >>> msgids = [make_msgid() for i in range(1000)]
<mwhudson> >>> len(msgids) - len(set(msgids))
<mwhudson> 5
 * mwhudson hated probability at uni
<mwhudson> if you take 10 samples with replacement from a population of 10000 what are the odds you'll get the same element twice?
<mwhudson> oh 100000
 * mwhudson tries an experiment
<thumper> damn
<thumper> that's really sucky
<thumper> relying on random for uniqueness is ick
<mwhudson> so in my experiment, the test will fail 50 times out of 100000
<thumper> or 1 time in 2000
<mwhudson> right
<thumper> I've seen it twice
<thumper> and we are up to build 535
<mwhudson> there may be reasons why it happens more often in builds
<mwhudson> (though i can't really think of any)
<mwhudson> looks just as sucky in python 2.6 too
<thumper> ew
<mwhudson> i'm surprised there isn't a python bug report for this
<thumper> mwhudson: so what do you think we should do about testing message id uniqueness?
<mwhudson> i don't know
<mwhudson> thumper: ellipsize the count?
<mwhudson> and add a big flaming comment
<thumper> well, remove the count, or make it a set of message id and to address
<thumper> mwhudson: I think it is just trying to count the emails
<mwhudson> thumper: i guess so yes
 * thumper writes some tests before running off for 2.5 hours
<mwhudson> thumper: i'm filing a python bug
<thumper> ok, cool
<thumper> mwhudson: I'm sure I've just broken a shed load of tests locally by making owner and product readonly
<thumper> running through ec2 to see how many :-|
<mwhudson> thumper: yeah, probably
<mwhudson> thumper: lots of easy fixes
<thumper> at the very least, branch +edit is broken
<thumper> hopefully
<thumper> although the edit form requires a bit of trickiness
<thumper> as the default behaviour of the notification is to take its own snapshot of the object
<thumper> and we'll have to do it locally first
<thumper> although I think I changed that code a long time ago to be smart :)
<thumper> ish
<thumper> mwhudson: I'm off now until about 3
<mwhudson> thumper: okidoke
<meaton2veggies> hi, sls
<meaton2veggies> hey all, im building launchpad and at stage of 'make scheme' which fails due to 'storm' not installing
<mwhudson> meaton2veggies: try bzr up download-cache
<meaton2veggies> mwhudson: update to date: rev 55
<mwhudson> meaton2veggies: then maybe try make clean; make ?
 * mwhudson has to run for lunch, biab
<meaton2veggies> mwhudson: error in output shows no Python.h in cextensions.c , im probably missing something for gcc?
<mwhudson> oh
<mwhudson> you need both python2.4-dev and python2.5-dev i think
<mars> yep
<meaton2veggies> ahk cool thats it cheers
<meaton2veggies> thought i had the dev pkgs n didnt
<meaton2veggies> thats clear
<mars> oh, the pain, right at the end:
<mars> ec2-register ec2-windmill-11//mnt/ec2-windmill-11/image.manifest.xml
<mars> /usr/bin/ec2-cmd: line 17: JAVA_HOME: JAVA_HOME is not set
<mars> ERROR:__main__:Oops!
<mwhudson> mars: heh, nearly there at least
<mars> mwhudson, do you know how to set the JAVA_HOME variable for subprocess.call() ?
<mars> without using env=, since that kills the shell environment
<mwhudson> mars: env = os.environ.copy(); env['JAVA_HOME'] = ..., call(env=env)
<mars> oh, awesome
<mars> thanks!
<mwhudson> np!
<mwhudson> on a syscall level, you always provide the entire environment aiui
<mars> hm. common pattern then?
<mwhudson> i guess
<mwhudson> it's what happens more or less when you do 'VAR=value foo' in the shell i think
<mars> ok, the script is finally done!
<thumper> mwhudson: hows the cvs bug hunting going?
<mwhudson> thumper: i've found a repo that fails to import from scrach finally
<mwhudson> thumper: rysnc-ing it to my machine now
<thumper> ok
<thumper> mwhudson: were you going to submit a testfix branch?
<mwhudson> oh crap, i was
<mwhudson> i didn't, though
<mwhudson> thumper: do you still want me to, or are you going to do it?
<thumper> mwhudson: if you are just waiting for an rsync, you can do it :)
<thumper> mwhudson: if you're busy, I could
<mwhudson> heh, it finished
<mwhudson> let me kick off a local import and i'll do it
<thumper> ok, ta
<mwhudson> thumper: https://pastebin.canonical.com/20576/
<thumper> mwhudson: r=me
<mwhudson> ta
<thumper> I'm unhappy
<thumper> the courier says they delivered my phone
<thumper> but it isn't here
<thumper> mwhudson: what would you suggest is the best way to get a member of a celeb team?
<mwhudson> thumper: !
<thumper> mwhudson: just grabbing the celeb team.owner?
<thumper> mwhudson: for a unittest
<mwhudson> thumper: an arbitrary member?
<thumper> yeah
<thumper> don't care
<mwhudson> yeah, owner sounds sane
 * thumper nods
<thumper> ok
<mwhudson> (owner is guaranteed to be a member?)
<thumper> I think so
<thumper> owner is considered a member for participation purposes
 * thumper is pretty sure anyway
 * thumper afk to get the girls from school
 * thumper has a new phone
<ajmitch> so the courier didn't lose it?
<thumper> no, he just put it around the back of the house next to the worm farm
<ajmitch> that makes perfect sense
<lifeless> is the owner always a member?
<spm> mwhudson: ta. I didn't like to assume transactions... :-)
<mwhudson> spm: all stuff in the webapp is inside a transaction
<spm> ah! good to know.
<_thumper_> I must get myself a surge protected nz multiboard
<thumper> plugged in my new phone
<thumper> sparky - reboot server - oops
<mwhudson> !
<thumper> spm: when is the edge update scheduled?
<spm> thumper: 0800 BST
<thumper> spm: can we get that changed back to my working day?
<spm> thumper: I replied internally, but you appear to be MIA?
<meaton2veggies> mwhudson: do u know all python deps for compiling/building launchpad - come across few compile errors when running 'make schema' i thought rocketfuel-get got all the deps?
<meaton2veggies> mwhudson: right now its failing at end of mailman compile
<mwhudson> meaton2veggies: the build process is a bit fragile
<mwhudson> meaton2veggies: pastebin the erros or i can't help at all though
<meaton2veggies> mwhudson: this one isnt clear, ill pastebin it
<meaton2veggies> mwhudson: http://pastebin.com/d411fa4a
<mwhudson> meaton2veggies: wow that is really unhelpful
<meaton2veggies> yes ha
<mwhudson> meaton2veggies: try make clean build ?
<meaton2veggies> mwhudson: yeah think i did, get same error when run 'check_mailman'
<mwhudson> i've no real idea, sorry :(
<meaton2veggies> is there a doc list of required python modules?
<meaton2veggies> still failing at end of mailman compile
<mwhudson> well, there's the launchpad-dependencies package, the list of things in versions.cfg and the list of things in utilities/sourcedeps.conf
<mwhudson> but you really shouldn't have to worry about any of those
<meaton2veggies> yeah i thought d/l sourcedeps
<wgrant> meaton2veggies: You used rocketfuel-setup?
<meaton2veggies> yes
<thumper> ah crud
<thumper> I can't have Interface as the definition for setTarget(target)
<mwhudson> thumper: tell me about it, i _still_ don't have a freaking test case for this cscvs bug
<thumper> nor can I have IBranchTarget
<thumper> as it has no canonical_url
 * thumper thinks more
<mwhudson> but otoh, i do know that if we throw away and restart a lot of the imports that currently have this problem, we'll find a lot of them start succeeding
<thumper> mwhudson: heh
<mwhudson> so i guess we should do that
<thumper> mwhudson: perhaps I should expose setProduct...
<thumper> and setSourcePackage
<mwhudson> on branch?
<thumper> setJunk
<thumper> yeah
<thumper> it is all shit
<mwhudson> :(
 * thumper is not a happy camper
<thumper> you know what...
<thumper> I'm thinking of just having ILaunchpadNamespace
<thumper> export that
<thumper> give it a canonical_url
<thumper> and be damned
<thumper> however... somewhere
<mwhudson> thumper: i think another option would be to have person, product, sourcepackage implement IBranchTarget directly, rather than via adaptation?
<mwhudson> that sucks too though of course
<thumper> nah
<mwhudson> uh, that's backwards
<thumper> no, because what would we say the interface is
<thumper> it's all bollocks
<thumper> perhaps sleeping on it will help
<mwhudson> oh right
<thumper> mwhudson: here's a new thing
<mwhudson> thumper: we're not having a very good week, are we?
<mwhudson> we need jono back clearly
<thumper> setTarget(product=None, sourcepackage=None)
<thumper> all the api calls need to pass by name anyway
<mwhudson> thumper: that works
<thumper> yeah
 * thumper does that
<thumper> tomorrow though
<lifeless> mwhudson: I feel your pain
<thumper> kids are fighting
<thumper> mwhudson: talk to you tomorrow
<thumper> night all
<mwhudson> thumper: later!
 * mwhudson goes to do some washing up, let's see if that helps
<jtv> hello antipodeans
<meaton2veggies> hi all, how do i get/install the launchpad-dependencies for ubuntu?
<meaton2veggies> looksl ike problems with the deps not being met mainly not having certain python2.4-* pkgs
<wgrant> meaton2veggies: rocketfuel-setup does it for you.
<wgrant> launchpad-developer-dependencies is the package, I believe.
<meaton2veggies> yeah rocketful should but some of the dep weren't installing
<meaton2veggies> installed some but not all due to some diff naming convention for pkgs
<meaton2veggies> on jaunty and its ppc port, also using internode au mirror i think
<meaton2veggies> installed remaining req pkgs manually instead of updating the launchpad meta-pkg
<meaton2veggies> wgrant: http://pastebin.com/d411fa4a
<meaton2veggies> wgrant: still getting this error, ne ideas?
<meaton2veggies> hi wondering if anyone could help disect this issue around building mailman - http://pastebin.com/m28dc3f72
<noodles775> Morning ppl
<noodles775> beuno: I'm hoping you're not really around, but if you are, and are in a state to give me more info on the navigation, that'd be great!
<al-maisan> Good morning!
<jtv> hi al-maisan~
<jtv> !
<al-maisan> hello jtv :)
<adeuring> good morning
<al-maisan> moin adeuring
<adeuring> hi al-maisan!
<wgrant> meaton2veggies: Oh, PPC. That would do it.
<wgrant> meaton2veggies: Jaunty and PPC probably won't work. Hardy and PPC might.
<wgrant> Wooooah.
<wgrant> Lots of broken sparklines.
<mrevell> Hi!
<wgrant> Morning mrevell.
<mrevell> hey there
<bigjools> helleau chums
<mrevell> :)
<jtv> oh hi guys!
<jtv> mrevell, did you find time to read through my text?
<mrevell> jtv: Hi! I got caught up in phone calls toward the end of the day, which is why I didn't respond to you before you signed off. It's next on my list. I'll PM you comments.
<jtv> mrevell: thanks.  Can I interest you in the latest version?  Very minor additions.
<mrevell> jtv: sure thing :)
<jtv> mrevell: a small but valuable life lesson, from one friend to anotherâwhen somebody asks "can I interest you in [...]," you ask about price first.  :-P
<mrevell> jtv: but I have done business with you in the past and so know what to expect :)
<jtv> Evasiveness, bad jokes & abuse.
<jtv> Oh, did I type that into the channel?
<mrevell> heh
<deryck> Morning, all.
<deryck> grrrr, those add/remove icons.  grrrr.
<beuno> noodles775, hi
<noodles775> Hi beuno :)
<beuno> noodles775, want to have a call about nav in a little while?
<deryck> beuno or sinzui -- so the fmt icon stuff now... there's a "modify" class to denote the remove icon?  Which is beyond the normal "add" and "remove" classes?
<sinzui> deryck: it is there in case someone need to know if the link changes an existing object.
<sinzui> deryck: there are no presentation rules for it
<sinzui> deryck: I did that so if beuno changes his mind, I do not need to be involved
<noodles775> beuno: yep, prolly after our standup... maybe in 1hr or so?
<beuno> noodles775, I've got a Team Lead call in an hour
<beuno> but maybe after that?  :)
<noodles775> beuno: or maybe now if you're free?
<beuno> noodles775, ok, let me get tea, and we'll do it
<deryck> sinzui, hmmm, ok.  So maybe it's the use of sprite-after that is different.  That class is no longer used?
<deryck> sorry, trying to work out what is different for a quick-strike fix.  ( and move on :) )
<sinzui> right. used...if you want the icon to follow the value of a field, we use fmt:icon
<sinzui> deryck: np, if this was easy, beuno would have given us the rules months ago
<deryck> sinzui, yeah, no problem.  I'm only grumpy about it because I want to move on from subscribing links :)
<sinzui> deryck: (-) unsubscribe {because the link text is not a value}
<sinzui> or Curtis Hovey (-) {because the link text is the value of my name}
<deryck> sinzui, gotcha.
<sinzui> I have decided we need to keep the NavigationMenu code and many of the menus because we need something to put the related page links in. We do not want to edit five pages to add/revise the related pages list every time something changes.
<beuno> noodles775, ready when you are
<noodles775> beuno: skype of voip? (skype tells me you're not logged in)
<beuno> noodles775, skype, calling you
<sinzui> bac, Edwin, salgado: stand up in 2 minutes
<beuno> noodles775, very nice improvements to the PPA page
<noodles775> :)
<noodles775> beuno: note, I'll remove the versions from the "Software updated by this PPA" portlet (as they'll be too long, and can be seen below in the list anyway)
<beuno> I think that's fine
<noodles775> Great.
<beuno> I'll look at it in more detail when the thread is over, but I think that it's improved massively
<noodles775> yup. thanks.
<abentley> beuno: what do you need for a UI review?
<beuno> abentley, if it's easy to try out on the UI, instructions for that. Otherwise, screenshots.
<abentley> beuno: Cool
<ursula_> hey stub
<ursula_> don't run :)
 * stub ducks under the desk
<Ursinha> lol
<Ursinha> stub, I see a few weird timeouts in lpnet
<Ursinha> stub, like OOPS-1307C157
 * Ursinha kicks the bots
<mars> that was ubot3.  he died.
<stub> Huh... never seen that before. No idea why that query would timeout.
<stub> I suppose it is querying a view on the replication machinery, so a big update could cause trouble. But we have had replication running very busy with large delays before without seeing that.
<Ursinha> I see 23 timeouts already in the partial report
<stub> Hmm... 0 seconds sql time. Does that mean all the time was chewed up before it attempted to query the database?
<Ursinha> maybe, or the oops report is borked
<stub> I guess we can't tell
<Ursinha> do you see a way to debug that?
<Ursinha> let me check the time they occurred
<stub> Well... we know it is borked because the query that times out doesn't end up on the statement logs with a nice starttime and endtime listed.
<stub> Until we fix that, I don't think we can tell if the time was spend earlier because the appserver paused or if the sql query took ages for some reason.
<Ursinha> I recall a bug to fix that
<Ursinha> stub, it seems all timeouts happened at the same time
<stub> Replication lag graphs say nothing unusual was happening. I have no idea what the trigger would be.
<Ursinha> stub, same exact time: 01:55:38 utc
<Ursinha> stub, I'll keep watching. In the meantime, I'll try to find the oops bug and see if someone is up to fix that
<stub> logs show nothing either.
<stub> Ursinha: Since you are there, https://code.edge.launchpad.net/~stub/launchpad/bug-354035/+merge/9419
<stub> I'm adding an extra field to the OOPS statement logs. I think some things will explode, so we should update them to cope.
<stub> What will explode I don't know ;)
<Ursinha> stub, awesome! I'll poke matsubara when he arrives today
<Ursinha> he'll know what would explode
<Ursinha> :)
<Ursinha> stub, I was talking about bug 310818
<mup> Bug #310818: Oops report does not always log timed-out query <Launchpad Foundations:New> <https://launchpad.net/bugs/310818>
<beuno> rockstar, hi
<rockstar> beuno, word gangster
<beuno> rockstar, how's it going?
<rockstar> beuno, just starting the day.
<beuno> :)
<beuno> rockstar, I think there's a bug witht he sparkline changes you made: https://code.edge.launchpad.net/launchpad
<rockstar> beuno, yeah, thumper wrote the code so that it generated those.
<rockstar> beuno, basically, I finished a thumper start.  I'll take a look at it later.
<beuno> rockstar, only the dev focus should have a sparkline
<beuno> and, they all display the same data
<rockstar> These sparklines are more work than they are worth...
<noodles775> sinzui: do you know anything about the <!-- :-) --> in base-layout? ;)
<beuno> noodles775, "mpt"
<noodles775> ah.
<sinzui> noodles775: it will go away
<sinzui> noodles775: it is a paste of the old layout issues
<noodles775> sinzui: well, I'm editing the facets/nav now, so I'll remove it now if you like?
<sinzui> noodles775: do it. it was a specific float issue in IE. Let's assume that YUI-reset as fixes this
<noodles775> k. Done.
<stub> The sparklines don't tell me anything I want to know, so seem like noise to me.
<beuno> unless, you're trying to figure out if a project is active or not
<beuno> that's the use case it's serving right now
<beuno> I've seen people use them that way many timees
<stub> The last commit date tells me that.
<beuno> no, it tells you the last time it was committed to
<stub> Yes. The sparkline gives me a false impression about activity.
<stub> Is a project with one 200k line commit in the last 90 days more active than a project with 10 commits every day averaging 15 lines each?
<stub> The answer still being - you don't know (because you don't know how meaningful those changed lines are)
<Ursinha> stub, herb, mars, rockstar, bigjools, henninge, sinzui, intellectronica: Production Meeting in 5 mins in #launchpad-meeting
<Ursinha> matsubara, ^
<stub> Of course, my opinion isn't a representative sample size ;)
<beuno> stub, yes, you can cheat the system. In general, regular commits means an active project.
<stub> Or a buggy one ;)
<Ursinha> stub, herb, mars, rockstar, bigjools, jtv, sinzui, intellectronica: Production Meeting in #launchpad-meeting now
<noodles775_> Grr... disconnects.
<noodles775_> <noodles775> beuno and sinzui: one thing is not clear to me about the templates you guys did for bazaar (showing the new 'facets' and breadcrumbs):
<noodles775_> <noodles775> We have the structured h1-heading that is part of base-layout, and there's a comment there that the future breadcrumbs will go below it, but looking at your mocks, I'm assuming that's where the facets go?
<noodles775_> <noodles775> Which leaves me wondering, will we be leaving it up to the individual pages to include the breadcrumbs below the h2 heading (as in your mocks)?
<noodles775_> ...or creating structured sub-headings.
<sinzui> noodles775: base-layout needs to do that. We should not be adding that to 400 tempaltes
<sinzui> I have a fix for the heading slot though. It is incompatible with the titleeditwidget. I hope to land it today
<noodles775_> sinzui: ok great. I'll wait until your branch is landed before continuing then. Thanks!
<noodles775_> sinzui: or is it safe for me to merge it now and continue?
<sinzui> noodles775: I it is a simple change to the h1
 * sinzui looks for the patch
<sinzui> noodles775: this is the change I expect to land: https://pastebin.canonical.com/20593/
<sinzui> noodles775: since the title edit widget must pass a <h1> element with special id and classes, *all* pages will have to do the same. :(
<noodles775_> sinzui: ok. That makes sense. But I'm still confused - am I right that it is the *facets* which will appear below that h1, not the breadcrumbs?
<noodles775> And if so, that we will need a similar structure for the separate subheading with the breadcrumbs, as in your mockup?
<sinzui> noodles775: I cannot say the design changes every day. I do not know what you are doing, I only know that beuno told me that the heading is going into the navigation, probably over the bread crumbs
<noodles775> sinzui: OK, but I'm just looking at your design at https://devpad.canonical.com/~curtis/LP_new_design_Bugs.png
<sinzui> noodles775: do you under stand now why you have not been able to make new pages...there is no approved design yet. You are doing this before beuno has gotten final approval
<beuno> sinzui, you've got mail
<beuno> hopefully clearing this issue up  :)
<sinzui> beuno: ^ is that design official? I do not think so
 * sinzui only offered to help, he does not have the power to choose a design
<noodles775> sinzui: I was asked to do the facets/breadcrums for the design (not that one specifically, one from beuno which is similar to yours, I juts don't have a url for it)
<sinzui> noodles775: you need to talk to beuno. I think you were asked without first being given scope
<bigjools> noodles775: just the tabs, not the breadcrumbs
 * sinzui looks at hirstory last picture he saw of design
<beuno> sinzui, I have a final design on navigation
<beuno> it's in your email
<noodles775> bigjools: yep, it's the tabs I'm doing now (sorry, didn't mean to include breadcrumbs).
<beuno> noodles775, one thing I forgot to mention
<noodles775> yup?
<noodles775> sinzui: sorry, I thought you'd be the person to check with (I was just confused by the comment about breadcrumbs in place of the nav in the base-template, but I guess it's there  just because it hadn't been settled at the time)
<beuno> is that the project/package/etc headings in the overview page are bigger font than they are in the internal ones, where the heading explaing what the page is, is bigger
<sinzui> beuno: nice shading. We should insist on that every time someone make an image with place holder content
<beuno> right
<sinzui> Well I see breadcrumbs under the heading in this design, so I think the comment is correct
<sinzui> oh, two heading
<noodles775> sinzui: yes.
<stub> mars: So I'm not sure what meaningful diagnosis we can do until we address https://bugs.edge.launchpad.net/launchpad-foundations/+bug/310818
<mup> Bug #310818: Oops report does not always log timed-out query <Launchpad Foundations:Triaged by stub> <https://launchpad.net/bugs/310818>
<beuno> noodles775, so, you see that "List of all open bugs" is larger than "Bazaar version control..."?   in the voewview page, that's inverted
<sinzui> beuno: is that a h2, or is the object.title demoted to a h2
<noodles775> ^^ my question exactly :)
<beuno> I'd say h2
<beuno> switch the h1 and h2 around
<beuno> maybe more symantically correct
<beuno> *maybe*
<salgado> sinzui, can you comment on bug 61171?  I've updated its description with my findings
<mup> Bug #61171: +login page OOPSes if query string has accented chars encoded as ASCII <oops> <Apport:New> <Launchpad Foundations:Triaged by salgado> <https://launchpad.net/bugs/61171>
<noodles775> But h2 before h1? Hmm... OK, I'll get back to you tomorrow with thoughts.
<sinzui> noodles775: beuno-lunch: that design does not show where edge/staging messages appear. the locationless layout probably needs some extra rules to work right
<sinzui> nice work salgado
<sinzui> salgado: do you want to check the user-agent to be certain that the bad url came from aport?
<salgado> sinzui, the user agent wouldn't tell me that -- apport opens the URL in the user's browser.  I know it's apport because I've seen the bug in the code
<sinzui> well then. I agree that this scenario is an upstream bug and we should not report an oops for it
<joey> anyone here and awake who is NOT a Canonical employee but is part of the community dev team (ie launchpad-dev) ?
<joey> I need a tester
 * maxb is
<joey> super maxb!
<joey> maxb: would you kindly try to edit this page and ensure you can save? https://dev.launchpad.net/FrontPage
<joey> maxb: just make any old change
<maxb> Based on my knowledge of moinmoin it should be sufficient for me to check whether I have an edit link or not  - which I do *not*
<joey> hmm
<joey> yeah, and I just got the error msg I was hoping not to get
<joey> ok, thanks maxb. I'll be back
<joey> maxb: please try now
<maxb> still no edit link
 * joey wrinkles his nose.
<joey> time for me to dig into the acl code :-(
<maxb> #acl launchpadTeamACL:read,write All:read
<maxb> Does launchpadTeamACL map to ~launchpad or ~launchpad-dev ?
<JamalFanaian> Hi, I'm having an issue getting launchpad running and I wanted to see if I could get some help.
<joey> maxb: yeah it should pull from the LP team however currently it's assigned to a page, not using the openid team code.
<kfogel> JamalFanaian: what problem are you encountering?
<JamalFanaian> When running make schema, I get an error about Missing download-cache (which seems to be there), and it tells me to run utilities/link-external-sourcecode.
<JamalFanaian> Running the script just asks me for an external source code, but I don't really know what it is I am suppose to supply to it.
<kfogel> hmmmm
<kfogel> JamalFanaian: you ran all the steps in Getting?
<joey> maxb: try logging out of the wiki and then back in please
<JamalFanaian> kfogel: Yes, as far as I know
<maxb> <joey> maxb: yeah it should pull from the LP team however currently it's assigned to a page, not using the openid team code.
<joey> maxb: and then edit
<JamalFanaian> I've gone through the instructions a few times to make sure I didn't miss anything
<maxb> joey: By "LP team" do you mean ~launchpad or ~launchpad-dev ?
<joey> maxb: ~launchpad-dev
<kfogel> JamalFanaian: it sounds like rocketfuel-setup must have encountered some error.  Could you move aside the previous stuff, redo form scratch, and save the transcript to show us?
<joey> maxb: I reverted the code a bit ago.. it's back again
<JamalFanaian> kfogel: ok i will do that, it will probably take a while though :)
<maxb> I've been logging in and out every time I tested - still doesn't show me an edit link
<joey> hmm
<joey> maxb: and you're in launchpad-dev yes?
<kfogel> JamalFanaian: sorry for that :-(.  I'm not actually a big expert in our build system, so I'm just trying to capture as much information as possible here -- then if I can't solve your problem, at least someone else can look at the transcript and know what is happening.
<JamalFanaian> kfogel: you don't have to apologize! it is helpful
<kfogel> JamalFanaian: also, I think a few other people have had this problem.  I want to identify it and then document the problem and its solution in the wiki; your transcript will be very helpful for that.
<JamalFanaian> what is the best way to save the transcript though?
<maxb> Yes I'm in ~launchpad-dev
<JamalFanaian> kfogel: should I just > the rocketfuel command to a file?
<kfogel> JamalFanaian: oh, hmm.  I do everything in an Emacs shell buffer, that works for me.  Do you run emacs at all?  Try M-x shell.
<kfogel> JamalFanaian: if not Emacs, then yes, do > into a file, and put 2>&1 on the end to get stderr as well.
<kfogel> JamalFanaian: then paste it to us at http://paste.ubuntu.com/
<joey> kfogel: can you try editing the front page of the launchpad dev wiki please?
<JamalFanaian> kfogel: alright doing that, thanks :)
<kfogel> JamalFanaian: if I'm not here when you finish, feel free to ping someone else, and if you solve it, PLEASE feel free to document the solution in the dev.launchpad.net wiki
<kfogel> joey: I could always edit it...
<joey> kfogel: try now :-)
<kfogel> joey: ah, I see yes.  here goes...
<kfogel> joey: bad news -- no Edit button for me.  I'm logged in.
<kfogel> joey: getting looooooooooonch now, bbiab
<joey> kfogel: ok, I obviously did something wrong. Let me file this spark lines bug and I'll go back to the code
<kfogel> joey: *nod*
 * kfogel is away: kfogel-food
<kfogel-food> dang it
<kfogel-food> I meant to change nick
<kfogel-food> let's try that again
<joey> maxb: kfogel-food - try now please
<maxb> no go, even after completely clearing my cookies for the domai
<maxb> n
 * joey grumbles. 
<deryck> beuno, ping
 * joey rereads the codes and has an idea.
<beuno> deryck, hi
<deryck> beuno, hey hey.  you can pull from those description editing branches of mine if you like; I've got updates based on our talks yesterday.
<beuno> deryck, doing now!
<deryck> beuno, there is a new image in the lazr-js one, so a make is required.
<joey> maxb: the code has a "you can't use a dash" caveat that seems to be part of my failing
<maxb> ah...
<beuno> deryck, got the lazr-=js link handy?
<deryck> beuno, bzr+ssh://bazaar.launchpad.net/~deryck/lazr-js/multiline-editor-ui/
<beuno> thanks
<deryck> np
<joey> maxb: ok again please.
<joey> maxb: fyi if you want to follow along - http://hg.moinmo.in/moin/2.0-storage/file/8f7a6efc77bf/MoinMoin/auth/openidrp_ext/openidrp_teams.py
<maxb> joey: still no :-(
<bac> salgado: this old wiki page https://launchpad.canonical.com/TeamParticipationUsage is mentioned in a few places in our tree.  any reason not to move it to the new dev wiki?
<bac> salgado: note i haven't read it yet...
<maxb> joey: Yikes. If I'm reading that right.... what a hack! :-)
<maxb> joey: So, which bit is breaking then? Is the raw text of the launchpaddevTeamACL page enlightening?
<joey> maxb: I don't have access to the logs. Elmo might if he's still in the office.
<maxb> *blink*
<maxb> Oh, so you don't have access to update the code?
<maxb> I'd assumed that's what you were playing with for each test
<joey> maxb: I'm just modifying the acl line
<maxb> Do you have read access to the operational moinmoin config?
<maxb> Unless someone has redefined page_group_regex, the problem could be that launchpaddevTeamACL does not end in "Group"
<joey> maxb: no that's locked down for security. I'm sure it's a malformed request on my end. I'll email the guy who coded it and seek his advice.
<beuno> deryck, dude!
<beuno> 99% perfect!
<beuno> fix the issue where the text moves 1 pixel on mouseover
<beuno> and ui=beuno + a hug
<deryck> beuno, ah, crap.  forgot about text when I moved it up to line with the header.
<deryck> will catch that now.
<beuno> deryck, then you just need a code review, and then nothing keeps you from making millions of users happy
<deryck> beuno, I have two issues otherwise, that have proved tricky so personally don't want to block on, but need your opinion...
<beuno> ok, shoot
<deryck> beuno, the width of the description is fixed.  Fluid proved tricky.  I assume that is okay. ?
<beuno> yes, I can live with that
<deryck> cool
<beuno> we can improve later if it turns out to annoy people. Scaling text works fine.
<deryck> beuno, and then, I the animation on the cancel icon only works on the first time.  after save or cancel it doesn't.
<deryck> without a page reload obviously.
<beuno> deryck, submit the code, and file a bug for it
<beuno> it is a bug, but not a big deal
<deryck> beuno, right.  I want to fix it, too, but to want to delay landing if its a couple days banging head on wall to work it out. ;)
<deryck> beuno, and potentially as I clean up the code these issues resolve, but if not, can get them in the next couple weeks after people play with this.
<beuno> I'm absolutely ok with that
<deryck> cool
<beuno> you've done an amazing job here
<deryck> beuno, thanks, man!
<salgado> bac, no, that page can be moved to the dev wiki
<bac> salgado: thanks
<bac> sinzui: ring me when you get a chance
<sinzui> bac: https://bugs.edge.launchpad.net/launchpad-registry/+bug/335509
<sinzui> bac: https://bugs.edge.launchpad.net/launchpad-registry/+bugs?assignee_option=any&field.assignee=&field.bug_commenter=&field.bug_reporter=&field.bug_supervisor=&field.has_cve.used=&field.has_patch.used=&field.omit_dupes=on&field.omit_dupes.used=&field.searchtext=&field.subscriber=&field.tag=&field.tags_combinator=ANY&orderby=-importance&search=Search&start=300
<sinzui> bac: https://bugs.edge.launchpad.net/launchpad-registry/+subscribe
<beuno> EdwinGrubbs, awesome landing of the picker replacements
<beuno> there is one behavior that seems to have changed
<beuno> if you type something in the input box, it doesn't get carried over to the picker
<EdwinGrubbs> beuno: thanks, I have a second branch that fixes some of the issues. I will need you to review some minor parts of it.
<EdwinGrubbs> beuno: that behavior had never been part of the picker. I can add it, but it will have to be in a followup branch, since my current branch is too big already.
<beuno> EdwinGrubbs, that's fine
<beuno> I'll file a bug for it, just thought I'd ask before I did
<mwhudson_> good morning
<thumper> morning
<thumper> I wonder how many emails I can get through in 5 minutes
<ajmitch> tag all, mark as read
<thumper> ajmitch: not the same...
<ajmitch> I know, I wish I could do that each day though
<_Groo_> hi/2 all
<_Groo_> devs, how do i change launchpad after initial install to be able to acess it remotely? im testing it with a vm
<_Groo_> web acess i mean, since bu default setup only adds entries to /etc/hosts
<_Groo_> anyone?
<ajmitch> setup DNS? I believe it should listen on all interfaces, not just localhost
<mars> _Groo_, you will have to modify the apache 'local-launchpad' config file, and open up the access beyond the current host
<_Groo_> mars: but i need to change all the 127.0.88 and .99 entries?
<_Groo_> mars: and corresponding ones in /etc/hosts?
<_Groo_> mars: ex: i have a virtual domain named lets say mydomain.com. what should i do so when i do a launchpad.mydomain.com i get launchpad?
<mars> _Groo_, not sure, actually.  You need to change the VirtualHost entries in the apache config, at least
<mars> and add "Allow" entries for other hosts (the default is to be closed to outside hosts)
<thumper> sinzui: https://lpstats.canonical.com/graphs/Branches/20080731/20090731/
<_Groo_> mars: yes but should i change the virtualhosts pointing to localhost, aka 128.0.0.88 and .99?
<mars> _Groo_, yep, change them to bind to '*', or a subnet
 * thumper looks around skype for his team
<_Groo_> mars: then i just restart apache or i need to ctrl-c make run and rerun it?
<thumper> mwhudson: skype ping
<mwhudson> thumper: i'm online
<mars> _Groo_, just try restarting apache first, and try visiting the launchpad.dev URL, see what the app server tells you.
<thumper> mwhudson: stupid skype
<mwhudson> thumper: yes
<mwhudson> thumper: i can restart if you can't connect, i guess...
<_Groo_> mars: i cant, lol, my virtual domain doesnt have X and links doesnt accept the ssl conection
<mars> :/
<mars> it doesn't like apache's self-signed cert?
<mars> did it work before?
<_Groo_> mars: first try :P
<_Groo_> mars: i just installed it via rocketfuel
<mars> ok, try running it locally, and visiting in lynx locally, before trying it remote
<mars> it is a development setup, not a ready-to-run product :)
<_Groo_> mars: yeah i know :D
<_Groo_> mars: a little wiki with some cmoon scenarios like this one would be nice
<_Groo_> mars: whats the default url to acess launch?
<thumper> spm: happy sys admin day
#launchpad-dev 2009-07-31
<Ursinha> oh, is it today?
<Ursinha> ah, for you thumper it is :P
<thumper> Ursinha: yes, you've got to remember I'm in the future :)
<Ursinha> thumper, indeed :)
<Ursinha> but, it is for spm as well :)
<thumper> true
<Ursinha> so, happy sysadmin day spm :)
<spm> thanks thumper, Ursinha, mwhudson!
 * spm adds priority flags to losa requests from those three....
<Ursinha> hahaha
<spm> ... errr for now. I may remove said flags at any moment and with no warning. I do have a bofhish reputation to maintain afterall!
<lifeless> jelmer: hai
<jelmer> lifeless: hey
<jelmer> lifeless: did you see my progress core review?
<lifeless> I landed it
<lifeless> did you see my progress-gtk branch?
<jelmer> I saw that there was a branch, but I haven't had time to try it yet
<lifeless> its pretty small if you could that would rock
<jelmer> I'll have a quick look
<lifeless> jelmer: hey, do you have perl subunit bindings for output ?
<jelmer> argh, the diff in lp is not against lp:subunit
<lifeless> resubmit it
<lifeless> it will make a new one
<lifeless> I'll do that now
<jelmer> I just did that, but now the diff is gone :-/
 * jelmer grumbles
<lifeless> give it a sec
<lifeless> there it goes
<lifeless> jelmer: ^
<jelmer> already reading it from a locally generated diff
<jelmer> (I wasn't aware it took a while for the diff to appear, I figured there was just something broken)
<lifeless> heh
<lifeless> its async
<jelmer> it would be nice if it could say "Updating diff.." like it does when it doesn't know a branches history yet, but knows it has been pushed to.
<lifeless> let me file a bug on that.
<lifeless> done
<jelmer> what's the bug #?
<jelmer> (I also filed one a few seconds ago, would like to mark it as a dupe)
<jelmer> never mind, brute forcing bug #s worked :_)
<jelmer> bug 407176 and bug 407177
<mup> Bug #407176: new reviews could say 'updating diff' <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/407176>
<mup> Bug #407177: indication that a diff is being generated on merge request page <Launchpad itself:New> <https://launchpad.net/bugs/407177>
<thumper> reviews are very much at the forefront of my mind right now
<mwhudson> heh heh heh
<mwhudson> er
<mwhudson> the bzr builder test run stopped because the appserver segfaulted!
<thumper> ouch
<wgrant> Yes.
<wgrant> Oops.
<mwhudson> thumper: FINALLY some progress on the cscvs bug!!
 * mwhudson runs off for a haircut
<thumper> \o/
<mars> thumper, have a moment to review a simple fix to the JS test suite driver? https://pastebin.canonical.com/20637/
 * thumper takes a look
<thumper> mars: r=me
<mars> cool, thank you
 * thumper is going to get the girls
 * mwhudson is back
<wgrant> mwhudson: Have you lost your CVS grokking ability with your hair?
<mwhudson> wgrant: well, i wouldn't be as cocky as to say that i grok CVS
<mars> mwhudson, ever set up a build slave image?
<mwhudson> mars: not from scratch, but i've updated them
<mars> hmm
<mars> ah, I would need a new one, from scratch, for the JS CI build stream
<mars> thanks anyway
<mwhudson> well, i wouldn't say 'from scratch'
<mwhudson> start from the existing AMI
<mwhudson> you'll need to fuck about on the master a lot too
<mars> well, no, probably based off of the devel image
<mwhudson> mars: probably best to do this with gary, he's in your tz after all :)
<mars> and yes, I would need to configure something in the master
<mars> heh
<mars> darnit, I am really close to getting JS continuous integration running
<mars> I just need to generate the image, and configure the buildbot system
<wgrant> How will that work? Run the browsers into xvfb?
<mars> yep: xvfb-run -s '-screen 0 1024x768x24' make jscheck
<wgrant> Useful.
<mars> oh yes :)
<mars> and 'make jscheck' in turn runs the entire LP windmill test suite
<mwhudson> mars: making the image will take a few hours at least
<mars> yeah :(
<mwhudson> mars: and it will be easier if you get some sleep, i promise :)
<mwhudson> mars: (it's nearly midnight for you?)
<mars> yes, it is :)]
<mars> firefox just crashed, taking 20 tabs with it
<mars> I think it was trying to tell me something
<mwhudson> seen, even your computer wants you to stop working
<mars> I would start playing nethack, but it would just nag me too
<mars> ok, good night :)
<mwhudson> night mars
<mwhudson> thumper: i think i have a fix for the cscvs bug!
<thumper> mwhudson: yay
<adeuring> good morning
<noodles775> Morning adeuring
<adeuring> hi noodles775!
<jtv> rockstar, you here?
<wgrant> jtv: Are translation commits meant to be done by codehost@crowberry?
<wgrant> That seems a bit ugly.
<jtv> wgrant: could you file a bug about that under rosetta please?
<wgrant> jtv: Sure.
<jtv> thanks
<jtv> abentley, you here?
<mrevell> Howdy
<jtv> hi mrevell!
<stub> BjornT: I can handle Bug #407288 if you can confirm I can throw the data away.
<mup> Bug #407288: BugNotificationRecipient needs pruning <Launchpad Bugs:New> <https://launchpad.net/bugs/407288>
<BjornT> stub: sounds good. i've commented on the bug.
<deryck> Morning.
<abentley> jtv: I'm here now.
<jtv> abentley: hi!  I ran into the fact that IBranchCollection.ownedBy doesn't return branches that the person in question is an _indirect_ user of.
<jtv> abentley: i.e, branches owned by teams that the user is in.
<abentley> jtv: I guess that makes sense.  I'm not sure how to get the list of branches you have write access to.
<jtv> I wanted your opinion on whether that is the right thing for it to do (in which case I should work around it e.g. by adding a method that casts a wider net), or if it should also include those branches (in which case it's basically not my problem  any more :-)
<abentley> jtv: I opine that both lists are useful, so we need two methods.
<abentley> jtv: For example, I'd expect ownedBy is used on code.launchpad.net/~abentley
<jtv> abentley: ah yes, that page goes through all teams you're in.  We had those timeouts with that this week.
<jtv> It counted branches owned by each of the teams you're in.
<abentley> jtv: In the main display, it only lists branches you personally own.
<jtv> abentley: maybe the best way to do this is to add an optional parameter for "indirect ownership too"?
<abentley> That might be the most discoverable for cases like yours.
<JamalFanaian> yess! launchpad is finally working :D
<JamalFanaian> well, ... make schema is working at lest
<kiko> JamalFanaian, woo, awesome
<kiko> JamalFanaian, want some bugs to try and work on fixing now?
<cjwatson> jtv: I was trying to figure out how to use http://blog.launchpad.net/general/exporting-translations-to-a-bazaar-branch with source package translations, but can't seem to see how to turn it on there. Is this supported for source packages yet, or only for projects?
<jtv> cjwatson: only projects for now, sorry
<cjwatson> ah, drat
<cjwatson> should I file a bug about it? I didn't see one
<JamalFanaian> kiko: uhh well sure! i don't know python *that* well yet.. so i was hoping to be able to get into playing with the project
<JamalFanaian> i can try to fix bugs but i don't know if i'm going to be too useful just yet
<kiko> matsubara, can you help find one or two trivials for JamalFanaian to work on?
<matsubara> sure thing. once we finish the call I'll look for some
<matsubara> JamalFanaian, are you interested in any specific are of Launchpad? Bug tracking, code hosting, translations, etc
<JamalFanaian> matsubara: not particularly, i wouldn't mind working on any part of the project :)
<rockstar> jtv, hi.
<matsubara> JamalFanaian, bug 126509 is very very trivial. just a text change in one of the templates and perhaps a test update.
<mup> Bug #126509: "Most Frequently Reported Bugs" is incorrect capitalization <trivial> <ui> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/126509>
<matsubara> JamalFanaian, would you like something more difficult?
<kiko> matsubara, it's a good first test :-)
<jtv> herb, you here?
<herb> jtv: yes sir
<jtv> great :) I wanted to ask for two things:
<jtv> herb: 1, is there any way I can see the output from the export-to-branch script?  It's not showing up on error-reports.  I'd like to see how the timing holds up.
<jtv> 2, there's something I'd like to test on staging later that'd involve a patch and a few script runs.
<jtv> herb: ah nm, tom just came back and gave me the info
<herb> jtv: ok. I'll nevermind then. :)
<jtv> sorry to bother you.  2 remains open though :)
<herb> oh. ok.
<herb> yeah. just let me know what you need for 2
<herb> I'm around for hours.
<herb> (with a break for lunch in there)
<JamalFanaian> matsubara: sorry i was gone for a few minutes.. no i'm fine with that.. it will help me figure out how TAL works and where templates are stored in launchpad
<JamalFanaian> kiko: heh, it is a good first test :)
<matsubara> JamalFanaian, all right. let me know if you need some help finding your way around the tree
<EdwinGrubbs> sinzui: did you send me the person page mockup?
<JamalFanaian> matsubara: thanks :) i will.. let me start playing with this, hehe
<mrkid> can sb help me?
 * sinzui does now
<mrkid> i try to follow https://dev.launchpad.net/Getting
<mrkid> but i get an error at ./utilities/launchpad-database-setup $USER
<mrkid> it's : -bash: ./ultilites/launchpad-database-setup: No such file or directory
<matsubara> JamalFanaian, sure. I'll leave for lunch in a bit. if you have any questions ask here and I'll help you out when I get back (or someone else might be able to answer)
<rockstar> mrkid, it looks like you're either not in the right directory, or you've actually mistype utilities.
<mrkid> i'm in the 'devel' directory
<mrkid> and i make a copy of this line into my terminal
<rockstar> mrkid, try `ls utilities`
<JamalFanaian> matsubara: alright thx :)
<rockstar> And pastebin that output, please.
<abentley> rockstar:chat?
<rockstar> abentley, I was just about to propose that.
<abentley> rockstar: ring me when ready.
<beuno> rockstar, would you like to do a UI review?
<rockstar> beuno, ABSOLUTELY!
<beuno> rockstar, I'll assign
<mrkid> rockstar, i tried but i got: administrator@ToshibaServer:~/launchpad/lp-branches/devel$ ls utilites
<mrkid> ls: cannot access utilites: No such file or directory
<JamalFanaian> oh gosh, i feel dumb.. i'm pretty lost.. is there any document that explains the directory structure of launchpad?
 * JamalFanaian thinks he is getting somewhere
<rockstar> JamalFanaian, grep is your friend.
<JamalFanaian> rockstar: heh, well i don't know what to grep for just yet
<JamalFanaian> but i found lib/lp which is what i was looking for
<rockstar> JamalFanaian, I suspect you want lib/lp/bugs/templates
<JamalFanaian> rockstar: for this its lib/lp/code/templates
<JamalFanaian> merge proposals are in the code section
<JamalFanaian> and the issue seems to be in the merge proposal template
<rockstar> JamalFanaian, which bug are you working on again?
<JamalFanaian> assuming i picked up the right bug
<JamalFanaian> #407347
<mup> Bug #407347: Failure detecting membership in the targeted review team - "(community)" label incorrectly showed <Launchpad itself:New> <https://launchpad.net/bugs/407347>
<JamalFanaian> and i didn't
<JamalFanaian> i have no clue where i got that bug number from
<JamalFanaian> went back through the look and saw that i was actually pointed to #126509
<mup> Bug #126509: "Most Frequently Reported Bugs" is incorrect capitalization <trivial> <ui> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/126509>
<JamalFanaian> oh gosh, that's a lot easier.. i was having a hard time figuring out how to replicate #407347
<mup> Bug #407347: Failure detecting membership in the targeted review team - "(community)" label incorrectly showed <Launchpad itself:New> <https://launchpad.net/bugs/407347>
<JamalFanaian> rockstar: i apologize for the misunderstanding, you are right :)
<beuno> jml, thanks for the feedback
<JamalFanaian> why would the "Most Frequently Reported Bugs" section not show up in my local environment? :\
<kedatinh> hello every body, i need some help
<kedatinh> when i try './rocketfuel-setup'
<kedatinh> i got error: Logging bzr into Launchpad (it's okay if this errors)...
<kedatinh> bzr: ERROR: The user name administrator is not registered on Launchpad.
<kedatinh> You can ignore any error above about not registering an SSH key
<kedatinh> with Launchpad.  Registering an SSH key is only important if you
<kedatinh> are writing data to Launchpad, or trying to access private data.
<kedatinh> make: *** No rule to make target `install'.  Stop.
<kedatinh> ERROR: Unable to install apache config appropriately
<kedatinh> so what can i do?
<noodles775_> kedatinh: it sounds like your bzr configuration has your launchpad_username as 'administrator'?
<noodles775_> (check ~/.bazaar/bazaar.conf)
<JamalFanaian> ok i have commited my changes and pushed them to launchpad, https://code.edge.launchpad.net/~jamalfanaian/launchpad/bug-126509
<JamalFanaian> do i just push a merge proposal for it now?
<rockstar> JamalFanaian, submit a merge proposal, but find a reviewer as well.
<JamalFanaian> do i just ask for a reviewer here?
<joey> maxb: time to help me with the wiki again?
<joey> maxb: I got some sysadmin help this time. :-)
<joey> mrevell: can you log out and back into the dev wiki please?
<salgado> sinzui, from our discussion about the context/menu TALES formatter expensiveness, I seem to remember that the biggest issue seems to be with items that may hit the DB for permission checking. is that right?
<sinzui> salgado: the biggest issue is that the list is recreated with each call, which causes duplicate db calls and permission checks. Once we start rendering a page, the state the the menu links can never change, so we should cache each call
<deryck> JamalFanaian, you go to #launchpad-reviews for reviews.
<mrevell> joey done
<joey> mrevell: ok. make sure you can edit
<joey> mrevell: maybe remove the "####" comment in the header
<salgado> sinzui, right, but if we had an easy way of caching the permission checks, thus avoiding the DB calls, recreating the lists wouldn't be a big deal, I think
<JamalFanaian> deryck: thanks
<deryck> np
<mrevell> joey: yup, no probs
<joey> mrevell: super. Community devs can now edit the front page
<mrevell> joey: Ah, superb :)
<joey> mrevell: kfogel - afaict, the front page is the only ACL locked page on the dev wiki
<sinzui> I see you point, but I do not understand why we think the link states will change from one call to the next. We know if they did change, there is a bug. The links cannot ever change once we call them the first time
<jtv> herb: a different request than expected...  could you "GRANT SELECT ON POFileTranslator TO translationstobranch" on the production db?  I've written up a branch that does the same for the long run.
<joey> mrevell: kfogel - so, we should be good there.  Matt, you'll want to make a decision about the ACls on the help wiki
<maxb> joey: Looks like you're sorted, but I'm around now.
<joey> mrevell: if you do want to make a change, it appears we need to make a config change to enable the new team, plus the new acl
<joey> maxb: super thanks. If you can log out and back into the dev wiki, you should be able to edit the front page now
<salgado> sinzui, right, I agree that it makes no sense to regenerate the list of links in a given request, but it might be easier to cache the permission checks than the list of links
<maxb> yes, I have an edit link now! :-)
<herb> jtv: done
<jtv> herb: thanks
<herb> welcome
<bac> hi sinzui -- have you pushed that announcements branch yet?
<sinzui> bac: I can push the branch. I had trouble extracting it from my other branch
<bac> sinzui: ok, i can wait.  untangling them would probably be best
<sinzui> bac: and this branch will depend upon a branch that I am testing before I submit to PQM
<bac> sinzui: thanks for the comments on those bugs.  i think i found all that can be handled during this effort.
 * bac -> lunch
<jtv> good weekend, everyone!
<mthaddon> herb: did you do that grant on all slave DBs too?
<herb> mthaddon: I did
<mthaddon> sweet
<mrevell> have a great weekend all
<sinzui> bac: you can merge lp:~sinzui/launchpad/announcement-edits, to get the navigationmenu changes
<bac> sinzui: thanks
<sinzui> I think I have found my first example of a browser test that is not testing what it claims.
 * sinzui considers changing it to a view test.
<JamalFanaian> matsubara rockstar: well it only took me 2 hours to finish that bug! -- is there another bug that you would want me to work on?
<rockstar> JamalFanaian, I'm sure I could find you some things...  :)
<rockstar> matsubara may be able to help you more unbiased than I though.  :)
<JamalFanaian> rockstar: o.O lol
<deryck> JamalFanaian, I think you can search launchpad projects for the tag "trivial" and find some simple fixes.
<JamalFanaian> deryck: alright i will do that, thanks
<deryck> JamalFanaian, np.  Thanks you for taking on some bugs! :)
 * maxb wonders if anyone has any thoughts on how to debug a twistd process that doesn't exit when you Ctrl-C out of "make run"
<rockstar> JamalFanaian, how 'bout bug 337582
<mup> Bug #337582: Branch sort fields say "by registrant name", but they mean "by owner name" <confusing-ui> <trivial> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/337582>
<matsubara> JamalFanaian, https://edge.launchpad.net/launchpad-project/+bugs?field.tag=trivial is a good start
<matsubara> JamalFanaian, also, when you start working on a bug, assign it to yourself and set the status to in progress
<matsubara> JamalFanaian, I just did that for you for bug 126509
<mup> Bug #126509: "Most Frequently Reported Bugs" is incorrect capitalization <trivial> <ui> <Launchpad Bugs:Triaged by jamalfanaian> <https://launchpad.net/bugs/126509>
<matsubara> JamalFanaian, and link the branch to the bug report
<JamalFanaian> matsubara: ah ok understood
<JamalFanaian> that makes sesne
<JamalFanaian> s/sesne/sense
<sinzui> beuno-lunch: ping
 * rockstar fetches a foods
<JamalFanaian> how do you guys usually deal when working on something else? do you revert your source back to the trunk?
<deryck> JamalFanaian, we do a branch for every bug or feature we work on.
<JamalFanaian> deryck: oh ok, i see there's a rocketfuel-branch script, let me try with that
<deryck> JamalFanaian, yeah, that's typically how we do it.  I think it just runs bzr branch and then links sourcecode deps from trunk.
<JamalFanaian> deryck: alright, thanks
<deryck> JamalFanaian, as your branches get more complicated, you want to keep them around even after review because you may have to come back to the problem, reference your work again, etc.
<JamalFanaian> deryck: that makes sense, i will do that
<JamalFanaian> it's funny how much i learned from changing 3 characters on my last bug lol
<sinzui> JamalFanaian: We usually assign new hackers trivial bugs during their first week because most of his time will be spent learning how to make the change.
<JamalFanaian> sinzui: yeah, it's actually a really great idea :)
<JamalFanaian> i mean, just from that i learned about running launchpad, where templates are stored and how launchpad is layed out, and how to run / modify page tests
<sinzui> right. your experience is the same for most of us. We are 100% distributed. We usually do not meeting someone for two weeks. At which time, we have lots of questions about why something does not work on our machines.
<matsubara> JamalFanaian, only mark a bug as fix committed when the branch has landed on lp:launchpad/devel
<JamalFanaian> matsubara: sorry i will revert that then
<matsubara> JamalFanaian, only mark a bug as fix committed when the branch has landed on lp:launchpad/devel
<matsubara> I'm having "An error occured when trying to install bzr 1.17" (full log: https://pastebin.canonical.com/20664/) when running make schema. Anyone got something similar and know how to fix it?
<matsubara> I'm running bzr 1.18dev
<maxb> I probably can't help anyway - but as that pastebin's not community-accessible I definitely can't help!
<kiko> matsubara, read carefully:
<kiko> bzrlib/_chk_map_pyx.c:31:18: error: zlib.h: No such file or directory
<kiko> matsubara, it basically says you're not on the diskless system. :)
<matsubara> should I remove my laptop's hard drive?
<mars> gary_poster, ping, have a pre-imq question for you
<gary_poster> mars: what's up
<mars> hi gary_poster.  I'm trying to get the 'test-all-windmills.py' script to have the lp dev paths configured
<mars> and that means either a) pulling the script contents into the existing bin/lpwindmill
<mars> or b) create a new script in bin/, bin/test_all_windmills.py
<mars> neither feels like a great solution
<mars> gary_poster, I was wondering what your opinion was
<gary_poster> mars: lp dev paths: do you mean like ppa.* and code.*?
<mars> gary_poster, I need eggs/windmill, eggs/mozrunner, etc.
<gary_poster> oh!
<gary_poster> mars: if I understand correctly, I would do what I think is your (b): I would move the script to bin, and have it generated by the buildout stuff.  You could do that using the setuptools approach or using that buildout-templates directory that is processed with z3c.recipe.filetemplate .  Do you not like (b) because it hides the script away in bin?
<gary_poster> (I must admit, I think it is more discoverable in bin in the abstract: it is a standard convention, and then you only have executables in there to dig through)
<mars> gary_poster, it is in utilities/ right now, and now I don't think that is right place for it, since it is an alternative test driver.
<mars> so bin/test-all-windmills.py sounds good
<mars> what is the buildout-templates approach?  setuptools is a bit much for now, as it is a single shell script that needs wrapping.
<gary_poster> mars, agree with utilities being odd
<gary_poster> mars: it's pretty easy.  You put templates (*.in) in buildout-templates, in a mirror of the dir you want (so buildout-templates/bin in this case).  When buildout processes the files, it substitutes in buildout bits and bobs.  There are plenty of examples in there.  Here are the docs: http://pypi.python.org/pypi/z3c.recipe.filetemplate
<gary_poster> mars: check out test.in, for instance
<gary_poster> It only does some processing at the top, and it is probably the exact same variables you want
<gary_poster> mars, actually we are using a slightly newer version; I think we are waiting on a review; need to check into that.  The newest docs are in eggs/z3c.recipe.filetemplate-2.1dev_gary_r101903-py2.4.egg/z3c/recipe/filetemplate/README.txt .  If this is bash shell script then you might want those features (ability to escape the {} syntax)
<mars> gary_poster, ok, thanks.  Should this go on the dev wiki, or is it a moving target?
<gary_poster> mas, you mean the fact that we use it?
<mars> just the process for adding something to bin/
<gary_poster> mars: oh...let me make sure I didn't describe it in the buildout doc.  Do you think it ought to go there?
<gary_poster> mars: and, no, I don't think this is a moving target
<mars> well, for buildout, there are two sources of information: how buildout works, and how to do things in our setup using it
<mars> "how buildout works" is on the PyPI page
<mars> "how do to things in our own setup", like an FAQ, is not around, afaik?
<gary_poster> mars: we have doc/buildout.txt for that purpose so far
<mars> ok
<gary_poster> mars, see the "Add a script" part of that doc
<gary_poster> mars, and "Add a File Modified by Buildout" beneath it
<gary_poster> It is described there as much as I thought was appropriate.  Looks ok to me.
<gary_poster> But I wrote it :-)
<mars> gary_poster, I think I'll make a tweak or two to the second paragraph, to explain the directory structure
<gary_poster> mars, awesome thank you
<mars> gary_poster, what does "In the Launchpad configuration, this is used in the ``[filetemplates]`` section" mean?
<mars> I don't understand how that sentence relates to using filetemplate
<gary_poster> mars: the [filetemplates] section of our buildout.cfg
<gary_poster> (the section could be named anything; we call it [filetemplates] instead of [yogurt] or [sillyputty])
<mars> ok, is that sentence necessary?  Or is it self-explanatory, by knowing how buildout works, and seeing the [filetemplates] recipe section in the .cfg?
<gary_poster> mars.  mm, the way I see it, this is all a compromise.  Everything is easier to understand zc.buildout, and the more you understand, the less you need any document like this one.  This is supposed to be a guide.  ...if you don't think it is helpful, I don't have any attachment to it. ;-) OTOH, I was *trying* to be helpful when I wrote it. ;-)
<mars> in other words, there is only one way to configure and activate a recipe, so saying "we use z3c.recipe.foo" doesn't need explanation as to how it is configured.
<mars> ok
<gary_poster> mars: to a hypothetical robot, agree 100%.  Less sure about a human.
<mars> heh
<JamalFanaian> So I am looking at this bug, https://bugs.edge.launchpad.net/launchpad-foundations/+bug/127489
<mup> Bug #127489: Answers column is in the wrong order in person's "Most active in" table <trivial> <ui> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/127489>
<JamalFanaian> and I've found that the view just sorts the categories alphabetically...
<JamalFanaian> now the issue is that i can't really figure out how i could set the proper sorting to go along with the application tabs without hard-coding that in :\
<salgado> JamalFanaian, it's possible to get the order of the tabs programmatically, but there's no way to map them back to the categories other than maintaining a hard-coded mapping
<abentley> rockstar: skype?
<salgado> I think the most reasonable thing to do is to hard-code the order in which the categories are returned, in PersonView.contributed_categories (which seems to be what's used in that page)
<JamalFanaian> salgado: yeah, that makes sense. i'm going about figuring out a proper way of doing that :)
<JamalFanaian> i'm fairly new to python and all, so it will take me a bit heh
<JamalFanaian> it's fun though
<salgado> and it will be lots of fun even after you become a python guru. :)
<JamalFanaian> salgado: hehe i hope so, so far i'm liking it a lot better than php
<Ursinha> abentley, hi
<abentley> Ursinha:
<abentley> Ursinha: Hi.
<Ursinha> abentley, I see you fixed bug 394800, but people reported bug 407347
<mup> Bug #394800: Members of review team shown as (community) <code-review> <Launchpad Bazaar Integration:Fix Released by abentley> <https://launchpad.net/bugs/394800>
<mup> Bug #407347: Failure detecting membership in the targeted review team - "(community)" label incorrectly showed <Launchpad itself:New> <https://launchpad.net/bugs/407347>
<abentley> Ursinha: Yeah, I've looked at that briefly, but it doesn't make sense.  It looks like some kind of fundamental flaw in the way we determine team membership.
<mars> rockstar, just saw the branch test run by in the windmill continuous integration suite - very nice work!
<Ursinha> abentley, so bug 394800 wasn't actually fixed?
<mup> Bug #394800: Members of review team shown as (community) <code-review> <Launchpad Bazaar Integration:Fix Released by abentley> <https://launchpad.net/bugs/394800>
<rockstar> mars, ?
<abentley> Ursinha: Matter of interpretation.  The rule is right now.  The things it depends on don't behave properly.
<mars> rockstar, first time I've run the code windmill test suite :)
<Ursinha> abentley, hm... right. could you triage the latter then, please?
<Ursinha> cprov, hi
<cprov> Ursinha: hi
<Ursinha> cprov, hi, just saw bug 378178
<mup> Bug #378178: Lauchpad tells me: "There is no package in Debian named "pidgin-plugin-pack". If it should be here, report this as a bug." - Though there is one! <Launchpad itself:New> <https://launchpad.net/bugs/378178>
<Ursinha> is that some kind of data corruption?
<JamalFanaian> Seeing as I have never used postgresql, is there a cli tool such as mysql that i can use to browse the db?
<JamalFanaian> nvm, found the answer myself
<JamalFanaian> postgresql-client :)
<cprov> Ursinha: well,  this package exists in debian -> https://edge.launchpad.net/debian/+source/purple-plugin-pack
<rockstar> mars, sorry, I guess I'm confused what I did that was out of the ordinary.
<mars> I just thought the popup dialogs looked nice - I like inline editing
<mars> the Bugs and Registry teams are doing great work too
<rockstar> Ah, so what you're saying is that you've never noticed my hard work before?  :)
<mars> I have, but I haven't used all of the features yet
<mars> running the full suite lets you see everything on the site in one go
<mars> there are quite a few JS interactions now!
<mars> really impressive
<rockstar> mars, yeah, I'm trying to get some more through right now.  Could you possibly do a code review for me?
<mars> rockstar, I would, but I really want to finish as much of the continuous integration as possible before going on leave
<rockstar> mars, alright.
<Ursinha> cprov, it seems the user is trying to open a bug to pidgin-plugin-pack, what isn't a valid package, it seems
<cprov> Ursinha: that's not what the form tells me.
<cprov> Ursinha: but that's probably because he has already filled an empty Debian task :(
<cprov> Ursinha: anyway, someone from the bugs team can provide you a better feedback, I'm not familiar with the code involved in this form.
<EdwinGrubbs> sinzui: is there an existing template converted to main_side? yui-g and yui-u aren't working as expected.
<sinzui> EdwinGrubbs: no
<sinzui> I have one that works,
<sinzui> EdwinGrubbs: This is a draft of a template that does layout correctly with the classes: https://pastebin.canonical.com/20672/
<sinzui> EdwinGrubbs: not that I have seen that "yui-u portlet" must be on the same div to work...and this is a problem with using the first because the portlet should demand to be first.
<EdwinGrubbs> sinzui: so you have to use two-column-list with <dl> elements? Is there a wiki page? I was looking at your email from last week.
<jms1989> hello, I have downloaded and installed the llaunchpad source code in a ubuntu 8.04 vm on a ubuntu 9.04 host. I want to view the web pages in a web browser on the host, how can I forward everything from the vm to the host so it would work?
<sinzui> EdwinGrubbs: yes, you do to get some presentations to work...namely term and value.
<EdwinGrubbs> sinzui: is there a wiki page?
<sinzui> No
<sinzui> The rules changed a few hours ago.
<sinzui> EdwinGrubbs: I will right a wiki page now that I think I know enough rules to give guidance
<sinzui> EdwinGrubbs: For example. I think we will have *more* NavigationMenus instead of ripping them out because they are needed for a lot of pages
#launchpad-dev 2009-08-01
<jelmer> lifeless: Hi
<jelmer> lifeless: Sorry, DebConf network went away yesterday
<mwhudson> wow, the new picker thingy is shiny
<wgrant> Does it work for branches yet?
<wgrant> Ah, it does. It sucks less than the old one.
<mwhudson> this is (a) true (b) not hard
<wgrant> Although it needs improvement.
<wgrant> "Please enter at least 3 characters."
<wgrant> How arbitrary.
<mwhudson> yes
<mwhudson> patches very very welcome in this area, i think :)
<wgrant> Mmmm. Why are the 3.0 announcement edit actions on the bottom, not the side?
<wgrant> (they are nice, though)
<jms1989> what would cause a ssl error when access the lp testing environment on 127.0.0.88 (launchpad.dev)?
<jms1989> I get (Error code: ssl_error_rx_record_too_long)
<wgrant> jms1989: Possibly trying to use the wrong port.
<jms1989> I am accessing it at https://launchpad.dev. that should automatically use port 443.
<jms1989> I'm not sure what I did wrong, I followed the instructions to get it working.
<jms1989> I'm using links and firefox to test it. both fail at the ip and domain level.
<wgrant> Possibly Apache segfaulting - try restarting it.
<wgrant> Just reloading after installing mod_python seems to cause problems.
<mwhudson> jms1989: maybe apache isnt set up right?
<mwhudson> i think serving plain http on port 443 will do that
<wgrant> That was my suspicion
<wgrant> But a similar thing can happen when the child segfaults, which happens if you rocketfuel-setup on a fresh Apache.
<mwhudson> really?
<jms1989> its a fresh install. think it was cause by the ubuntu installer installing lamp and ssh server at the install stage?
<mwhudson> rocketfuel-setup doesn't activate mod_python, does it?
<wgrant> wgrant: It doesn't? Hm/
<wgrant> Maybe it was another module.
<wgrant> Er.
<wgrant> mwhudson: ^^
 * wgrant looks harder.
<jms1989> how can
<mwhudson> jms1989: try sudo apache2ctl restart, read the error.log and if that doesn't work pastebin /etc/apache2/sites-enabled/local-launchpad
<jms1989> I find out if mod_python is enabled?
<wgrant> I certainly had it segfaulting after rf-setup reloaded it.
<wgrant> restarting fixed it.
<jms1989> restarting apache says that /var/tmp/bazaar.launchpad.dev/static/ and /var/tmp/ppa doesn't exist.
<wgrant> That's fine.
<wgrant> The latter is my fault, but it shouldn't be a problem.
<jms1989> http://jms-bin.pastebin.ca/1514582
<jms1989> hmm, after redoing the build process, I'm getting an error saying qrunner isn't running but the line above is says it shutdown qrunner. I'm confused.
<wgrant> That's normal.
<wgrant> For me it then proceeds to start everything up.
<jms1989> logging out and restarting the process didn't work. ssl still fails. Id I access http://127.0.0.88/ directly, I get a page saying nothing exist on that address.
<jms1989> ttyl, going to bed.
#launchpad-dev 2009-08-02
<ppzico> Hi, I have a question about packaging. I have a single project having 2 different executables. Another is made with C and the other with Python. Those executables communicate through TCP/IP on localhost.
<ppzico> How should I package them?
<ppzico> Maybe as multi binary or should I consider having those executables different projects?
<wgrant> ppzico: You probably want #ubuntu-motu.
<ppzico> ok thanks
<wgrant> ppzico: This is for development of the software that runs launchpad.net.
<ppzico> I see, sorry for bothering here
 * wgrant vomits.
<wgrant> mwhudson: BeautifulSoup to render a view!?
<mwhudson> wgrant: not my fault!
<gcerquant> I'm reading the documentation at https://dev.launchpad.net/Getting to install Launchpad on my own machine. I have seen that right now, it can only be done on Ubuntu. I'm wondering if anyone has patched it to install it on a Mac.
<gcerquant> Any link to a blog post, or tips to do it?
<thumper> morning
<mwhudson> thumper: hi
<thumper> mwhudson: call?
<mwhudson> thumper: yeah
<mwhudson> thumper: i think the ec2 image did actually start for the db_lp builder
<thumper> ah, ok
<thumper> it is stuck somewhere else?
<mwhudson> i guess
<mwhudson> hm
<mwhudson> traceback on slave: http://pastebin.ubuntu.com/244720/
#launchpad-dev 2010-08-02
<lifeless> garh
<lifeless> this search stuff is so frustratsing
<wgrant> Hm. Is loggerhead running?
<thumper> \o/ my conflict resolution branch just landed
<james_w> there's a lot of confusion between tests in lp.archiveuploader and some of those in soyuz
<wgrant> I moved a whole lot a couple of days ag.
<wgrant> But yes.
<wgrant> It's pretty bad.
<james_w> which way did you move them?
<wgrant> I moved doctests from soyuz to archiveuploader.
<spm> wgrant: seems to be working for me? no alerts, and a test looked good. ??
<wgrant> I left soyuz-upload.txt alone, though, since it's not clear where it should live.
<wgrant> spm: Odd, was 502ing for me almost immediately.
<james_w> wgrant: can you see http://ec2-75-101-196-90.compute-1.amazonaws.com/summary.log?
<wgrant> james_w: No.
<lifeless> james_w: you need to open the firewall more, manually.
<james_w> wgrant: http://paste.ubuntu.com/472006/
<james_w> that confused me for a while, the second isn't actually at the path that it says it is
<james_w> it's in soyuz, and it appears to test exactly the same thing
<wgrant> That's one I just moved.
<wgrant> Sure you're up to date?
<james_w> plus I just wrote a bunch of tests for checkUpload, and it now turns out that it is tested in archiveuploader
<james_w> ah, that's probably why, I'm probably based of an older devel
<wgrant> Right. It used to live in archiveuploader.
<wgrant> Then it was factored out.
<james_w> ah
<james_w> the tests were changed but not moved I guess
<james_w> how frustrating
<wgrant> Have you seen some of those doctests?
<james_w> yes
<james_w> these were unit tests though
<wgrant> Ah.
<jelmer> james_w: Sorry, that must be my fault.
<jelmer> The way that code worked was a bit messy, where an Archive method called out to a plain function and then function then called out to the Archive's methods again. I cleaned that up (by merging the logic into Archive) but never got around to moving the tests along.
<james_w> jelmer: I think I can move all of lp.archiveuploader.tests.test_permission in to lp.soyuz.tests.test_archive?
 * jelmer looks
<jelmer> james_w: yep
<james_w> great
<lifeless> I have renewed my hate relationship with doctests
<lifeless> also with layering, fragile tests and DB's.
<lifeless> wgrant: so, on this private librarian issue
<lifeless> wgrant: I think we are, today, completely insecure.
<lifeless> wgrant: by which I mean many things that are on private objects are on the public librarian
<wgrant> lifeless: Right. Only P3As really use the restricted librarian.
<wgrant> Oh, and MP diffs.
<lifeless> I'm wondering how risky it is, to do it without C-D, and add C-D in later
<lifeless> oh, other angle
<lifeless> all restricted librarian content can attack all othe restricted librarian content at the moment too
<lifeless> in that it can get project names, urls, do same-site scripting.
<lifeless> thumper: ping (private librarian usage)
<lifeless> thumper: can we make the stuff we put in the librarian for private MP's be totally preformatted ? then the client could access it directly.
<wgrant> lifeless: Restricted librarian is purely proxied.
<wgrant> And the proxied stuff is always C-D'd if dangerous.
<wgrant> In fact, it may always be.
<wgrant> I forget.
<lifeless> wgrant: I'm pretty sure that that is a porky
<wgrant> The restricted librarian is not accessible from the outside.
<wgrant> Only via the webapp.
<lifeless> yes
<lifeless> thats rather my point
<lifeless> its in the LP domain
<lifeless> lib/canonical/launchpad/browser/librarian.py
<lifeless> like 141
<lifeless> we set C-E
<lifeless> and C-T
<lifeless> and thats it.
<wgrant> Yes, I just found that.
<wgrant> That's broken.
<lifeless> so
<lifeless> I'm saying:
<lifeless>  - do we add C-D:attachment now
<lifeless>  - or get direct access now and add it later
<lifeless> I'd rather not do both direct access *and* add C-D, because of the potential for interactions and confusion.
<lifeless> though clearly, we want to end up doing both (or doing something crazy like uuid DNS names to isolate content more.
<wgrant> I guess that it's no worse than now as long as it's served separate from the public librarian.
<lifeless> wgrant: yes, we'll want a new hostname for sure.
<lifeless> is private.launchpadlibrarian.net going to be good enough ?
<lifeless> or will it need to be yet-another-sibling ?
<wgrant> No.
<wgrant> launchpadlibrarian.net is full of untrusted content.
<wgrant> Wait.
<wgrant> That might be safe, then.
<thumper> lifeless: re private librarian, we use the same diff for both email and web usage
<thumper> lifeless: so, no, not easily
<lifeless> thumper: so to fix your timeout you have two choices.
<lifeless> fix the librarian client bug that it doesn't set/use a timeout
<lifeless> or make the content you store immediately deliverable
<thumper> ââ¹
<thumper> WTF!
<thumper> bzr is lying
<lifeless> spm: hey
<lifeless> spm: nevermind
<thumper> or should I say ec2 test is lying
 * thumper wonders how...
<wgrant> I have a success mail from EC2 from two hours ago.
<wgrant> No sign of it in PQM yet, though...
<thumper> wgrant: I have a branch that it "merged" into devel
<thumper> wgrant: but failed the build step
<thumper> wgrant: on a line that isn't in the resulting branch
<wgrant> Nice.
 * thumper tries again
<lifeless> poolie: ping
<lifeless> spm: how is rt 39643 ?
<poolie> hi lifeless
<poolie> what's up?
<lifeless> poolie: how is the featureflags stuff ? Can I encourage folk to use it, or is there more needed before it can be adopted ?
<thumper> :(
<thumper> buildbot all read
<thumper> red
<lifeless> poolie: I realise its not-all-finished
<poolie> they can use it now
<poolie> i don't expect to pull anything out
<poolie> it will be easier when i land the thing currently rated tweak
<poolie> oh, i think it's only in db-devel at the moment
<poolie> so that may limit things a bit
<poolie> (this would incidentally have been a pretty safe change to apply onto the live db
<poolie> since it's just creating a new empty table
<lifeless> I think the table exists
<lifeless> if it doesn't, you can ask stub to do it, and then merge your code to devel
<poolie> so next up is:
<poolie> add some scope detection for beta users etc
<poolie> web api to tweak it
<poolie> logging
<poolie> better way to test things tat depend on features
<thumper> lifeless: how to I merge in the reverse of the last revision?
<poolie> but none of these should block use now
<thumper> lifeless: to back out a change
<lifeless> thumper: merge -r X..X-1 .
<thumper> lifeless: ta
<spm> lifeless: a place of suffering and pain is where it's at, unf. am hoping to grab Mr K shortly for some thoughts and guidance on reducing the pain
<lifeless> spm: cause, I'm told that getting that done is blocking my two fav lp rt's
<lifeless> 40477 and 40480
<lifeless> which in turn are blocking making *everyone* a lot happier.
<spm> that happens
<StevenK> spm: Moi?
<spm> indeedily; but shoo - lunch. grab you after.
<StevenK> Hehe
<lifeless> spm: when you get back, I would like 5-10 minutes to talk roadmap of deployments etc
<lifeless> not to change anything, just to get me on the same page
<spm> lifeless: sure. I'm back, it was SK heading lunchwardly.
<lifeless> oh
 * lifeless checks the clock :)
<lifeless> spm: so, whats in the losa pipeline for lp
<spm> pain, misery suffereing. the usual.
<lifeless> spm: have you seen - https://dev.launchpad.net/LEP/ReleaseFeaturesWhenTheyAreDone
<spm> not in a final form like that; but was aware of the broad outline
<lifeless> ok
<lifeless> this is going to be a pretty big shift, I want to tie it in nicely with your workload
<lifeless> AIUI the main thing on your plates is the lucid migration, which involves pg8.4 too
<spm> for LP, yeah pretty much
<lifeless> how deep is that pipeline, realistically ?
<spm> in terms of?
<spm> just LP or everything?
<lifeless> are we there yet :)
<lifeless> hmm
<lifeless> I'm trying to assess the following:
<spm> heh, I wish. we haven't even done staging as a trial run
<lifeless>  - are we blocked on manpower on the operations side to effect the changes desired
<spm> tho we have done LS successfully; some gotchas in that; and at least one issue that may (I stress may in the unknown sense) be a blocker for LP/U1 et al
<lifeless>  - is there *anything* the developers need to do that isn't yet done, for the pg8.4, lucid, and RFWTAD changes
<spm> RFWTAD ?
<lifeless> https://dev.launchpad.net/LEP/ReleaseFeaturesWhenTheyAreDone
<spm> oh right, of course.
<lifeless>  - if there is nothing developers can do, and we're not blocked on manpower, then I can start asking folk to use the new process; otherwise I need to rally something on one side or the other.
<lifeless> or if we're blocked on decisions, technical input, etc
<spm> for pg8.4/lucid - no idea. I have no visibility into what has been done/tested there. so NFI if it'll work or not. ??
<spm> that we don't have a working test setup for lucid doesn't bode well in the first instance
<lifeless> I think there is a certain swings-and-roundabouts aspect there
<lifeless> it's been made critical though, AIUI
<lifeless> ok, so thats 10 minutes - thanks, we're cooked for now :)
<spm> yeah, so's everythingelse (critical) tho. :-(
<lifeless> critical on the dev side
 * poolie off to lunch
<poolie> lifeless: happy to talk about flags later if you like
<lifeless> coolio
<poolie> i wasn't planning to work on it for the next couple of days other than to tweak & land what's already there
<lifeless> we're about to enter freeze, I think
<poolie> i wanted to work out how to make  the docs visible
<lifeless> go have tood
<poolie> perhaps putting them in a module docstring would be best
<lifeless> *food*
<lifeless> we can talk when you return
<poolie> late food
<poolie> :)
<poolie> ok
<lifeless> spm: also, is there an RT for daily-staging ? I think that that may have slipped through the cracks.
<spm> lifeless: ? we already update staging every day; several times a day infact
<lifeless> yeah, names are terrible.
<lifeless> this is staging-with-production-schema
<spm> there's no difference. ??
<lifeless> yes there is
<lifeless> rt 40482
<spm> as in; if we just rollout code to staging without coresponding DB schema changes; staging simply won't start.
<wgrant> Isn't staging with production schema just edge?
<lifeless> wgrant: can't do destructive testing on edg.e
<wgrant> Oh, right.
<lifeless> wgrant: there is /no/ place that you can QA edge properly.
<lifeless> wgrant: and thus, edge does not get QA'd.
<spm> oh! right. sorry - my bad. Prod schema. not prod+1 schema.
<spm> lifeless: given how little used staging is atm; is there any value in having another QA env? edge has the advantage of at least being used by lots?
<lifeless> spm: QA != users using it.
<lifeless> spm: see the RT - same hardware.
<lifeless> and we do, until we delete db-devel, require two environments because we have two revisions that can be deployed: db-stable:tip and stable:tip
<spm> lifeless: one minor ~ish nitpick. the staging db only restores once a week. it takes something like 20+ hours to happen.
<lifeless> hah
<lifeless> well, $same-schedule-please
<spm> yup :-)
<wgrant> But you can surely upgrade the schema without restoring it fully.
<lifeless> wgrant: we don't want to upgrade the schema
<wgrant> Um, yes, ignore me.
<lifeless> thumper:  File "/var/launchpad/test/lib/canonical/shipit/browser/shipit.py", line 73,
<lifeless>                                                        ^^^
<lifeless> bah
<lifeless>                                                                      ^^^
<lifeless> better
<lifeless> thumper: shipit is out of tree now, so you probably need to import the errors from the old location, fix shipit, and then do a three-branch landing dance.
 * lifeless takes a break
<wgrant> Can't we just stop rolling out LP to shipit?
<wgrant> I don't see why we still do it.
<lifeless> does it have any data in common now ?
<wgrant> It doesn't use Person any more.
<wgrant> So I don't think so.
<wgrant> Except for Account, which is what we want to destroy.
<wgrant> Oh, no.
<wgrant> Looks like it does still use Person: for checking shipit-admins membership, and confirming karma.
<poolie> lifeless: oh i meant to ask, is there going to be an architectural overview, and if so where?
<poolie> are yougonig to build on the one bjorn started?
<lifeless> poolie: no
<poolie> (i'm not assuming it needs to be all your work)
<poolie> uh
<lifeless> I think we want to improve the docs
<poolie> so things people should/might want to know before hacking on launchpad will go where?
<lifeless> for now, where they go at the moment - the launchpad.dev wiki
<lifeless> having many different places to look would add confusion, not reduce it.
<lifeless> I rather like what bzr has, and a low-pri task would be to start working towards that
<lifeless> right now its sufficiently far down the problem-scale that its unprioritised, at least from my perspective.
<poolie> wiki is fine with me
<poolie> i think it's just good to have a centre of gravity
<lifeless> for your flags stuff, I would like the following written up somewhere.
<lifeless> what should LOSA's know
<poolie> so you can say "that should go in X" and gradually get the habit
<lifeless> what should developers writing code know/think of
<lifeless> what should maintainers-of-the-module know/think of
<lifeless> and possibly, what should users know (we might want users to see their active flags?)
<poolie> mm
<poolie> advanced users perhaps
<lifeless> I think having a good module docstring is really nice.
<poolie> right, especially if that's built onto the web somewhere
<poolie> then the wiki can point to that
<lifeless> I think it is on the apidocs
<poolie> right
<lifeless> but not https://launchpad.net/+apidoc/devel.html - thats different again.
<lifeless> there is a zope thing
<lifeless> anyhow, if someone can do 'pydoc lp.services.flags', I think thats pretty cool, but it doesn't really help with discoverability.
<lifeless> https://dev.launchpad.net/Hacking is the centre of gravity for 'getting started doing something in LP'
<poolie> i'll put something on the wiki and link to the api docs
<poolie> i'm glad someone built them to html
<poolie> (gary?)
<poolie> we just need to make that public to non-canonical people
<wgrant> poolie: You mean pydoctor output?
<poolie> mm
<poolie> i thought it was on chinstrap or similar
<wgrant> http://people.canonical.com/~mwh/canonicalapi/
<poolie> that's great
<poolie> http://people.canonical.com/~mwh/canonicalapi/lp.services.features.html ! :-)
<poolie> wgrant: has that always been there?
<lifeless> poolie: since mwh put it together 3-4 years back :)
<poolie> ok so salgado said about gary's
<poolie> > Unlike the other docs we have, these ones know about interfaces and the classes
<poolie> which implement them, adapters, views, global utilities, zcml directives
<poolie> and a few other things, so go check it out.
<wgrant> Oh, Zope apidoc?
<wgrant> Yeah.
<poolie> it would still be nice to move that
<wgrant> That's on devpad.
<wgrant> But accessible easily from a local instance.
<poolie> which is private?
<wgrant> devpad's private, yes.
<thumper> lifeless: ah... I've been fixing something in a symlink :(
 * wgrant wonders what lp.soyuz.xmlrpc is about.
<wgrant> It's empty, and Soyuz has never done XML-RPC.
<lifeless> thumper: is your git-test fixing fix landed ?
<thumper> lifeless: yes
<lifeless> \o/.
 * lifeless tries to land his malone patch AGAIN
 * wgrant ponders a massive c.l.i extermination branch.
<StevenK> I wonder if apidoc.launchpad.net is on the cards
<wgrant> Unlikely. It provides source access.
<StevenK> So does launchpad.net/~launchpad-pqm/devel ?
<wgrant> They like to pretend that we don't have access to shipit.
<wgrant> Apparently giving away free CDs is a risk?
 * thumper EODs
<lifeless> wgrant: huh ?
<wgrant> lifeless: Well, it's somewhat irritating that I have to revert to devel r8967 to see if I can safely delete something. That's when shipit was removed -- which I really don't see the benefit of.
<wgrant> "Oh no, we can't have them giving away free CDs."
<lifeless> wgrant: was it perhaps back in the 'not all will be open sourced' period ?
<wgrant> lifeless: It was at the same time SSO was (somewhat less inexplicably) removed.
<wgrant> And around the time they were considering omitting most of Soyuz.
<lifeless> so
<lifeless> you could propose a merge back in :P
<wgrant> Well, I'd prefer that the split was finished.
<wgrant> The only benefit at the moment is that it can see karma.
<lifeless> sadly the split is along fairly nuts lines
<lifeless> if you wanted to make it use the API to do what it does, that would be great.
<wgrant> Apart from that, it could live in an entirely separate DB with a static copy of LP.
<wgrant> lifeless: How is the split 'along fairly nuts lines'?
<lifeless> vertically rather than front/back end
<wgrant> Well, ShipIt could sanely be a separate application.
<wgrant> SSO not so much.
<lifeless> ok, -> awol for a while
<adeuring> good morning
<jkakar> lifeless: Hi!
<jkakar> lifeless: If I remember correctly __nonzero__ was explicitly not added to ResultSet because it can cause performance problems (though, right now I can't remember what those exact reasons are).
<lifeless> it is equivalent to try: memo = self.any() except IsEmpty: return False, isn't it ?
<lifeless> (mumble details)
<wgrant> lifeless, stub: Thanks for the reviews.
<mrevell> Hello
<wgrant> bigjools: Hi. Should I get the last ddeb DB change landed this cycle (the checkbox on Archive:+admin to enable ddeb building)? Although it works OK for PPAs, it won't really be safe to turn it on anywhere until 10.09, I suspect, since the primary archive handling isn't done yet.
<bigjools> wgrant: yes that's fine as long as the feature is turned off
<bigjools> wgrant: we should consider moving that checkbox to +edit though?
<wgrant> OTOH I could make the copier reject DDEB copies to primary, then it would be fine for PPAs.
<bigjools> there's only one team that can copy from a PPA to primary
<wgrant> bigjools: Probably, yes. But we might need to work out expiration policies and the like before we make it widely available.
<wgrant> Moving the checkbox can be done at any time; adding the DB column can't.
<wgrant> Although PPA binaries are expired really quickly anyway.
<bigjools> yes, it should not be enabled until everything works :)
 * bigjools -> OTP
<wgrant> k
<wgrant> lifeless: I've seen .count() == 0 around. But that's probably far less efficient than .any().
<bigjools> ResultSet needs __nonzero__ then we can use bool(blah)
<wgrant> But it's possible that it's omitted for the same reason __len__ is.
<wgrant> Callers assume it's cheap.
<bigjools> __nonzero__ lives on SQLObjectResult though
<bigjools> I've learned not to assume anything when dealing with Storm code now :)
* bigjools changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 2  of 10.08 | PQM will be closing 22:00 UTC Friday | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
* bigjools changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 3  of 10.08 | PQM will be closing 22:00 UTC Friday | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<wgrant> Hm.
<wgrant> http://blog.ganneff.de/blog/2010/08/02/removalstxt---removals822.html might make it easy enough to implement relibale deletion support in gina.
<lifeless> StevenK: hi
<lifeless> StevenK: why do you propose things then move them to WIP immediately ?
<wgrant> bigjools: https://code.edge.launchpad.net/~wgrant/launchpad/multi-arch-builders will let us switch amd64 builders to i386 through the LP UI, letting admins easily alleviate situations like the current crisis... it removes the master's arch check, sends the build architecture to the slave, and the slave respects that.
<wgrant> Any objections?
<wgrant> (also handy for when LP learns about processor compatibility...)
<bigjools> wgrant: yes I object - until lamont looks at it, because I know it needs work on the slave.
<wgrant> bigjools: Well, of course.
<bigjools> but yes I'm sure he'll enjoy any branch that makes his life easier
<wgrant> Although the i386 builders have been gone for nearly four days now.
<wgrant> Is something up with them?
<wgrant> Unless alpha 3 is, uh, reallllly slow...
<lifeless> thumper: I still see <ImportError: No module named shamap> from ec2
<thumper> lifeless: which revision of devel did your testrun use?
<wgrant> jelmer: Hi. Do you have a PQM email for that branch you ec2'd for me earlier? I got an ec2 success email, but it never made it through PQM...
<lifeless> bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/, revision 11272
<lifeless> thumper: ^
<thumper> lifeless: hmm... that should have the reverted sourcecode change
<thumper> lifeless: not sure what is going on there
<lifeless> me neither
<lifeless> oh
<lifeless> I see, my bad, I had some overlapping stuff going on
<lifeless> thumper: sorry for the noise.
<thumper> ack
<StevenK> lifeless: Because bigjools is lazy and wants to look at my branch
<bigjools> oxymoron
<StevenK> lifeless: So rather than pulling down my branch, he likes me to use a WIP MP
<lifeless> StevenK: you can tell lp-propose to start it in WIP
<bigjools> can you create them as WIP, I think he means
<StevenK> Oh, right
<lifeless> you *will* get reviews from me otherwise
<StevenK> I use the web UI to create them
<bigjools> lol
<lifeless> and from anyone else tracking the feed
<lifeless> StevenK: try clicking on advanced
<StevenK> lifeless: Yes, so noted
<wgrant> It'd be nice if everything had a WIP MP.
<lifeless> wgrant: its call 'a branch'
<wgrant> lifeless: Yes, but I can't get a diff unless I grab it or sacrifice some goats to loggerhead.
<bigjools> guffaw
<wgrant> It never works the first time.
<wgrant> I don't know why.
<wgrant> It sometimes works on the second refresh. And normally on the third.
<StevenK> lifeless: Actually, you should look at that MP
<StevenK> lifeless: *Why* is dsilvers effectively running SQL by hand, and surely there is a better way?
<lifeless> storm :)
<lifeless> and collections
<StevenK> lifeless: Mind looking at the code before you say so? ;-)
<lifeless> sure, I'll look tomorrow am
<bigjools> lifeless: this was before Storm days, I'm not sure if the same methods are needed any more
<wgrant> I don't know a way of doing such mass inserts in Storm, though.
<bigjools> there isn't - store.execute() FTW :)
<wgrant> Well, except for just doing a direct SQL -> Storm conversion.
 * lifeless hates on the testfix glue
<jml> lifeless, yeah, we should do something about that :)
<thumper> jml: hi
<jml> thumper, hello
<thumper> jml: big branch today moving errors
<thumper> running through ec2 now
<lifeless> jml: check my kanban
<jml> thumper, yay
<lifeless> why is PQM rejecting non-testfix when its also building ?
<thumper> jml: most of the bigness however was changing imports :-|
<lifeless> losa ^ ?
<jml> lifeless, https://dev.launchpad.net/Trunk/Glue might have a hint
<mthaddon> lifeless: I think there needs to be a successful build again to get it out of testfix
<jml> thumper, why's that bad?
<thumper> jml: not bad, just a lot of boring changes
<lifeless> mthaddon: I thought it was 'branch started' was all that was needed.
<lifeless> thumper: https://lpbuildbot.canonical.com/builders/db_lp/builds/977/steps/shell_7/logs/summary
<jml> thumper, *nod*
<mthaddon> lifeless: I don't think so - I'm not sure how it would be aware of that
<lifeless> mthaddon: its what the wiki page said
<jml> thumper, I've been meaning to challenge someone to write an emacs equivalent to that neat vim import statement thing EdwinGrubbs showed at the epic
<jml> thumper, that'd make it less boring.
<jml> (or, you know, write it myself).
<lifeless> mthaddon: 'PQM is put in testfix mode if either branch is unclean, where "unclean" means "the last test run failed and we haven't had a testfix revision or a forced build yet".'
<lifeless> thumper: do those failures look related to the merge conflict you fixed ?
<thumper> lifeless: could be
<thumper> lifeless: but I don't think so
<lifeless> so, could be spurious
 * lifeless kicks it
<thumper> lifeless: the conflict I fixed was source package recipe build related
<mthaddon> lifeless: so has someone sent in a testfix branch?
<lifeless> thumper: the only change in the blamelist is yours though
<lifeless> mthaddon: yes
<lifeless> mthaddon: I think the machinery is probably working
<thumper> lifeless: there may have been other db commits before mine
<thumper> lifeless: that would have been skipped in blaming
<lifeless> will be -so- glad when db-stable is no longer used.
<thumper> lifeless: like https://lpbuildbot.canonical.com/builders/db_lp/builds/977/steps/shell_7/logs/summary
<thumper> oh
<thumper> that failure is from before my merge
<jml> lifeless, I'm still hanging out for mandatory pre-merge testing, like in the good ol' days
<lifeless> jml: thats planned too
<lifeless> jml: I was very happy when gary suggested it
<lifeless> In fact, now that my search branch passes all tests
<lifeless> if I can just get the damn thing landed, I'll be hacking towards premerge testing
<lifeless> thumper: so, what does this mean ?
<thumper> lp.buildmaster.tests.test_buildqueue.TestBuildQueueDuration.test_current_build_duration is almost certainly soyuz commit related
<thumper> nothing in the sprb related to that
<thumper> lp.code.browser.tests.test_sourcepackagerecipe.TestSourcePackageRecipeBuildView.test_eta is probably related, but missed due to changes in both branches
<thumper> lib/lp/buildmaster/tests/../doc/builder.txt, no idea, but I'd look at soyuz
<lifeless> thumper: so, is it worth kicking it, or does it definitely need a further patch ?
<thumper> lifeless: it'll need fixing
<thumper> they aren't spurious
<lifeless> bigjools: ^
<bigjools> what?
<lifeless> bigjools: can you, or your nominee, fix db-devel :)
<lifeless> it appears to be fallout from some soyuz changes
<bigjools> have you got a buildbot log?
<lifeless> yes, up above
<bigjools> ta
<lifeless> https://lpbuildbot.canonical.com/builders/db_lp/builds/977
<lifeless> https://lpbuildbot.canonical.com/builders/db_lp/builds/977/steps/shell_7/logs/stdio
<bigjools> this smells of abentley
<bigjools> did it pass devel and fail on db-devel?
<lifeless> there was a merge conflict on db-devel, which thumper fixed this morning
<lifeless> when it finished chewing on it, we got the above.
<bigjools> hmm
<lifeless> thumper reckons it wasn't the merge made it go boom
<thumper> boom happened before conflict resolution
<bigjools> ok
<thumper> but I didn't notice
<thumper> otherwise I would have fixorated it too
<thumper> and now I'm not working
<lifeless> indeed
<thumper> trying to get a proposal in for kiwipycon
<lifeless> and I have a call tomorrow morning
<lifeless> thumper: when is the deadline
<thumper> 1.5 hours
<lifeless> hah
<thumper> 1 hour 24 minutes
<thumper> to be precise
<bigjools> wgrant: the failing code is yours
<lifeless> well, you can tell them 'sorry but I need to sleep'
<wgrant> bigjools: Icanhaslog?
<bigjools> https://lpbuildbot.canonical.com/builders/db_lp/builds/977/steps/shell_7/logs/stdio
<wgrant> EPERM
<bigjools> shut
<lifeless> thumper: ^ :P - I can try to put something together tomorrow am if they are interested.
<bigjools> shit, even
<bigjools> hang on
<lifeless> gnight y'all
<bigjools> wgranthttp://pastebin.ubuntu.com/472143/:
<bigjools> sigh
<bigjools> http://pastebin.ubuntu.com/472143/
<bigjools> nn lifeless
<wgrant> bigjools: Um.
<wgrant> Interesting.
<bigjools> recipe_bq.processor = i386_family.processors[0]
<bigjools> failing
<bigjools> processor is not in set_attributes
<bigjools> so either the layer is wrong or the zcml is wrong
<wgrant> Or somebody started proxying them recently.
<bigjools> ha
<bigjools> right
<bigjools> that's it
<wgrant> I haven't touched that code lately.
<bigjools> it should be in the set_attributes
<wgrant> Why?
<wgrant> It's immutable.
<bigjools> hmmm good point
<wgrant> rSP ftw.
<bigjools> sigh
<bigjools> that stuff should not be proxied in that test
<bigjools> it runs zopeless ffs
<wgrant> Zopeless is not Zopeless :(
<bigjools> well, you know what I mean
 * bigjools fixx0ors
<bigjools> I'd like to know how it passed tests to get in there like this
<wgrant> I don't know what the incompatible db-devel change could be.
<bigjools> me neither
<bigjools> factory returning proxied objects but why only on db-devel
<wgrant> Exactly.
<wgrant> bigjools:
<wgrant> -        return bq
<wgrant> +        return ProxyFactory(bq)
<wgrant> Ahem.
<wgrant> (it was the merge)
<bigjools> whut
<bigjools> which merge?
<wgrant> thumper's conflict resolution.
<wgrant> db-devel r9603.1.2, in particular.
<bigjools> presumably that came from devel
<wgrant> 9603.1.1 was the merge and conflict resolution
<wgrant> .2 was "Fix the failing tests"
<wgrant> So it suggests not, but I haven't looked closely.
<bigjools> hmm
<thumper> oops
<thumper> that was just to shut the warnings up
<bigjools> oops
<bigjools> your fix is ok, it's exposed a bug in the test
<bigjools> but I think we should revert that fix and file a bug about the bug
<bigjools> thumper: sound ok?
<deryck> Morning, all.
<bigjools> wotcha deryck
 * bigjools can't fathom why someone would run builder.txt in LaunchpadFunctionalLayer
 * bigjools submits testfix
<bigjools> thanks for spotting that wgrant
<wgrant> np
<bigjools> errm, why am I getting a warning about converting from 2a to KnitPack5 when pushing a branch...
<wgrant> Can someone please land https://code.edge.launchpad.net/~wgrant/launchpad/refactor-nuf-creation/+merge/30851? I believe it was rejected in the testfix this morning.
<jelmer> wgrant: Sure
<jelmer> wgrant, it was indeed
<wgrant> jelmer: Thanks.
<bigjools> Using saved push location: lp:~julian-edwards/launchpad/db-devel-testfix
<bigjools> Doing on-the-fly conversion from RepositoryFormat2a() to RemoteRepositoryFormat(_network_name='Bazaar RepositoryFormatKnitPack5 (bzr 1.6)\n').
<bigjools> wth
<wgrant> bigjools: You had an old branch sitting around.
<bigjools> grar
<wgrant> Yeah, it's private, so it's nice and old.
<bigjools> not any more it's nto
<wgrant> That's one way to fix it.
 * bigjools would really love it if we could have an alias for the submit-branch value
<jml> bigjools, thumper has a db-submit bzr alias
<bigjools> aliases are great, until you don't have them on a different machine
<maxb> This is why your home directory should be in version control :-)
<jml> or, you know, Ubuntu One
<wgrant> Maybe OneConf will happen at some point.
<jelmer> a gconf backend for ubuntu one would be awesome
<bigjools> when there's a U! client for KDE, I'll use it
<bigjools> U1, even
<wgrant> It'd need to be an overlay or something equivalent, but yes.
<sinzui> hi jcsackett
<jcsackett> hi sinzui.
<jcsackett> and hello bac and hello EdwinGrubbs
<bac> hi jcsackett!
<danilos> losa ping: I don't see any emails explaining the status of our buildbots, can someone please clarify why's db-lp offline
<danilos> and does anyone else know why's xvfb failing to start on lucid builder: https://lpbuildbot.canonical.com/builders/lucid_db_lp/builds/57/steps/shell_7/logs/summary
<mthaddon> danilos: I've been reconfiguring things in preparation for lucid switch (we now have a lucid-devel builder) - I suspect it just needs a restart
<mthaddon> danilos: if a "force" fails, that is
<danilos> mthaddon, I've already tried a force on the lucid-db-lp builder
<danilos> mthaddon, the previous one failed in the same way: https://lpbuildbot.canonical.com/builders/lucid_db_lp/builds/56/steps/shell_7/logs/summary
<maxb> If you have a moment, mthaddon, could you append the missing letter 'l' to https://edge.launchpad.net/squirrelmai ? :-)
<mthaddon> danilos: ok, can we wait til lucid-devel and lp are finished, then we'll do a restart
<danilos> mthaddon, certainly
<mthaddon> maxb: done
<maxb> thanks :-)
<mthaddon> np
<james_w> bigjools: the bug in checkUpload would be that it would never tell you that you had no permissions for the archive, just that you didn't have enough permissions. (No-one would ever see the "Did you mean to upload to a PPA?" error)
<bigjools> james_w: ah!!!
<bigjools> that's, errr, interesting
<james_w> could have been a lot, lot worse
<bigjools> and explains a lot
<bigjools> yeah
<jml> hmm.
<jml> I guess poolie's DKIM thing hasn't landed yet?
<james_w> what layer should I use for tests that don't need anything at all really?
<james_w> no database, just running python code
<salgado> james_w, no layer. :)
<james_w> perfect :-)
<maxb> i386  	3  	 1139 jobs (five days)
<maxb> *sob*
<beuno> maxb, the rumour is that there are a few more build slaves on their way
<maxb> We can only hope rumours materialize soon :-/
<lifeless> moin
<jam> Ursinha: didn't you want to submit something to bzr-pqm a while back? I haven't seen any merge proposals
<jam> morning lifeless
<lifeless> hi jam
<Ursinha> jam, I'm working on it today, just busy with other lp duties
<jam> Ursinha: no problem. I just thought you already had something and I just didn't see the submission
<Ursinha> jam, ah, you can be sure I'll poke you when that happens :)
<lifeless> flacoste: https://dev.launchpad.net/LEP/ReleaseFeaturesWhenTheyAreDone
<deryck> Later on, all...
<lifeless> https://lpstats.canonical.com/graphs/OopsLpnetHourly/
<lifeless> flacoste: lpnet - 1177927, edge - 94787
<jelmer> sinzui, hi
<sinzui> hi jelmer
<jelmer> sinzui: It looks like python-pocket-lint still isn't installable on hardy - in its current form it needs python2.6 and a recent version of python-support
<sinzui> yuck
<jelmer> sinzui: I'd like to fix the package and reupload - do you have the source for the previous revision in a branch somewhere?
<sinzui> I think I can make some quick changes to make it python 2.5 compatible
<sinzui> jelmer, https://bugs.edge.launchpad.net/pocket-lint
<sinzui> jelmer, the packaging branch has the debian dir
<jelmer> sinzui: ah, thanks
<sinzui> There is a 5.1 release expected to build in the next 9 hours
<jelmer> sinzui: Is there a particular reason gnulog is shipped with pocket-lint, or just because it's convenient?
<sinzui> I like gnulog for generating logs. It is a habit from by gnome
* lifeless changed the topic of #launchpad-dev to: Launchpad Development Channel | Performance day! | Week 3  of 10.08 | PQM will be closing 22:00 UTC Friday | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<lifeless> losa ping
<lifeless> losa unping
<jelmer> sinzui: I've pushed a branch with a few packaging improvements and a fix to support installation for all system-provided pythons that are 2.5 or newer.
<sinzui> I see
<jelmer> sinzui: Would it be ok if I pushed a build of this to the Launchpad PPA with a version number lower than what you're building at the moment? That way I can still attempt to build a new EC2 image but my build will be gone once you copy the packages in from your PPA.
<sinzui> jelmer, your mp just disappeared
<sinzui> I was going to approve it. Yes you can change the version to keep working
<jelmer> sinzui: Sorry - https://code.edge.launchpad.net/~jelmer/pocket-lint/packaging/+merge/31587
<poolie> hi sinzui, jelmer
<jelmer> 'morning Martin
<sinzui> hi poolie
<wgrant> OMG BUILDERS!
<sinzui> jelmer, your branch is now tip
<jelmer> sinzui: Thanks!
<jelmer> wgrant: 'morning!
<wgrant> Morning jelmer. Thanks for landing that.
<jelmer> You're welcome. I see you have more branches in store for us though.
<wgrant> Yeah, a few.
<jelmer> thumper: 'morning
<thumper> jelmer: hey, your branch yesterday had failing git import tests
<jelmer> thumper: Hmm, that's odd :-/ I'll see if I can fix that here.
<wgrant> Ahem.
<wgrant> Did Bugs really just start streaming arbitrary user files through the webapp?
<wgrant> Yes, yes they did.
<elmo> wgrant: err, what?
<wgrant> elmo: Around an hour ago, a rev landed on db-devel to stream private bug attachments through the webapp.
<wgrant> With no C-D.
<elmo> only private bugs?
<elmo> not that it helps much
<wgrant> It's reusing existing code which should probably be sending C-D anyway, but it's previously only really been used for LP-generated files or .debs.
<wgrant> Only private bugs at the moment, it seems, yes.
<lifeless> so
<lifeless> C-D should be added
<lifeless> wgrant: care to put up a trivial branch? rs=me
<wgrant> It's going to change some Soyuz behaviour (no more inline build log viewing for private archives).
<lifeless> I've run my crazy dns idea past flacoste and he didn't cry
<lifeless> so we might head towards that
<lifeless> wgrant: you could add a variant class that does C-D
<lifeless> wgrant: ^ hi?
<wgrant> lifeless: Hi.
<wgrant> Hmm, maybe.
<jelmer> thumper: Hmm, I can reproduce the git import test failure. I'm at a loss as to what's happening though and what my changes could have to do with it. Do you have any clues?
<thumper> jelmer: was it the import failure?
<jelmer> thumper: yeah, I'm getting two test failures in the git import tests
<jelmer> thumper: LINE 1: ...ccount = Account.id AND LOWER(EmailAddress.email) = E'no-pri...
<jelmer> ProgrammingError: operator does not exist: text = bytea
<jelmer> is that the same error you saw?
 * thumper is looking for the errors
<lifeless> jelmer: I saw 'shamap' import errors
<thumper> jelmer: https://lpbuildbot.canonical.com/builders/lp/builds/1119/steps/shell_7/logs/summary
<thumper> jelmer: the 6 errors are all shamap import failures
<thumper> File "/srv/buildbot/slaves/launchpad/devel/build/lib/lp/codehosting/codeimport/tests/test_worker.py", line 1018, in tearDown
<thumper>     from bzrlib.plugins.git.shamap import mapdbs
<thumper> jelmer: did you remove that module?
<jelmer> thumper: yeah, I renamed it.
<thumper> jelmer: so fix the tear down and it should be good
<thumper> jelmer: I'm not sure what your other failures are from
<jelmer> I might see if I can reproduce that one on my work laptop.
<jelmer> Anyway, I'll have a look at the tearDown failure first and get that fix through ec2.
<thumper> jelmer: I did a reverse merge of your change to devel
<thumper> jelmer: so you'll need to reverse my reverse :)
#launchpad-dev 2010-08-03
<jelmer> thumper: Will do. Thanks.
<mwhudson> jelmer: man, that mapdbs fix had me seriously head-scratching :-)
<mwhudson> i hope it's not necessary any more
<jelmer> mwhudson: heh, I bet
<jelmer> mwhudson: it's still used actually
<jelmer> though we'll hopefully be able to use bzr index files with git pack files for the cache hopefully
<mwhudson> ok, maybe more of a fix than deleting the tearDown is required
<jelmer> (that should also be the point at which we can have a "bzr serve --git" that works with decent performance)
<jelmer> that reminds me.. any news on bzr:// support, or is it all recipes these days in the code team ? :-)
<thumper> jelmer: all recipes
<lifeless> jelmer: bzr:// support - for anonymous access to branches ?
<jelmer> lifeless: yeah
<lifeless> jelmer: I'd rather we just did bzr on http
<lifeless> jelmer: less setup, better support in firewalled situations
<jelmer> lifeless: either would be awesome
<lifeless> wgrant: btw
<lifeless> wgrant: private PPA log file tailing. That permits an attack on launchpad.net via content insertion, no ?
<wgrant> lifeless: Howso? logtail should be escaped by the template.
<lifeless> ah
<lifeless> I was thinking put js in it and let the streamorview thing do its sniffing
<wgrant> And the downloable build log should be text/plain.
<wgrant> Ah, no, there's no JS there.
<wgrant> Just boring old refreshing.
<lifeless> and the build log is the full thing, right ?
<wgrant> It is.
<wgrant> There's probably a vulnerability there somewhere.
<lifeless> wgrant: you were saying that setting cd:attachment will break streaming logs
<lifeless> wgrant: but if its being escaped in the appserver, it won't have any immediate impact, will it ?
<wgrant> lifeless: Not streaming logs. Viewing of logs in the browser.
<wgrant> At the moment I can click on a build log link and not have to download it.
<wgrant> This is unrelated to logtail.
<lifeless> ok
<lifeless> anyhow, I've filed a bug, if you have ideas about how to do things, please dive in.
<wgrant> I think the existing build log streaming might be vulnerable in IE, since IE likes to sniff content rather than obeying content types.
<lifeless> heh
<wgrant> How much SQL time is there on OOPS-1675ED4744?
<lifeless> is that oops code correct ?
<mwhudson> wgrant: did it just get created?
<wgrant> Maybe it just missed the sync.
<lifeless> SQL time: 4761 ms
<lifeless> Non-sql time: 13888 ms
<lifeless> Total time: 18649 ms
<lifeless> Statement Count: 59
<wgrant> Right, that's about what I thought.
<wgrant> WTF is the non-SQL time, I wonder.
<thumper> wgrant: python execution time,
<thumper> storm object creation time...
<lifeless> we can get a profile of it for you
<lifeless> if you ask a losa
<wgrant> I know, but that's an awfully large amount of time considering the few queries.
<lifeless> pasting you some stuff
<lifeless> losa ping
<spm> yo
<lifeless> I'd like to get a profile from staging per favour
<spm> sure
<lifeless> so, in theory I wait, you say 'go ahead' then I say 'done'.
<lifeless> in practice, I'm guessing this is the first time ever and you have no idea whats about to happen :P
<spm> that'd be spot on
<lifeless> and the difference between theory and practice.. :)
<lifeless> so in the config file for staging
<lifeless> on staging
<lifeless> there is a profiling section
<spm> I can infer what you want; but not quite sure exactly what you want to achieve, and the how of.
<lifeless> with a key profile_requests (IIRC)
<lifeless> you need to turn that on and bounce the app server
<spm> huh. I didn't know that!
<lifeless> tada
<spm> this is the app server? (to confirm) vs code*
<lifeless> we're going to make this less manual
<lifeless> appserver
<lifeless> https://api.staging.launchpad.net/beta/~ubuntu-dev/participants is the URL I want to profile
<lifeless> spm: for context, see maris's email to launchpad-dev at the end of the epic
<lifeless> spm: so, 2 weeks backish. it describes the process on a developer machine.
<spm> hrm. we have no 'profiling section' or 'profile_requests' is the existing configs; can you give me a quick dump of the lines to add?
<spm> s/is/in/
<lifeless> [profiling]
<lifeless> profile_requests: True
<spm> hmm. complex.
<lifeless> (see configs/schema-lazr.conf for missing things like this)
<lifeless> you can also set profile_dir:
<lifeless> if you want them to go to a /var dir or some such - the default is '.'
<spm> what!?!? srsly!?!? you want me to rtfm!?!?! I'm a *sysadmin*!!!
<lifeless> oh, and lastly, we should also set
<lifeless> *in that section*
<lifeless> error_dir: and oops_prefix:
<lifeless> because it will generate an oops to go with, so that we get the DB trace etc
<spm> ahh kk
<lifeless> https://pastebin.canonical.com/35303/
<lifeless> to bring it all together.
<spm> not such a big deal here'n'now; but might be worth getting this formally landed into the prod configs - staging section only. ??
<lifeless> yeah
<lifeless> if you tell me the stuff to put in it :)
<spm> staging-lazr.conf ?
<spm> oh. right. lala eparse
<lifeless> I can wait if you need a coffee :)
<spm> [profiling]
<spm> profile_requests: True
<spm> error_dir: /srv/staging.launchpad.net/staging-logs/profiling
<spm> profile_dir: /srv/staging.launchpad.net/staging-logs/profiling
<spm> oops_prefix: PROFILE
<spm> do I what...
<spm> losa_needs_coffee: ZOMGTRUE
<spm> restarting the app server...
<spm> Kah Boom.
<spm> https://pastebin.canonical.com/35304/
<lifeless> HAH
<lifeless> delete the error_dir key
<spm> kk
<lifeless> oh
<spm> same error.
<lifeless> delete the oops prefix too
<lifeless> the schema had a confusing bit, it got me good
<spm> :-)
<spm> that's happy
<spm> see that's what rtfm does for you. confuses. much better to just make s**t up
<lifeless> is it up and running ?
<spm> shouldbe... stopped doing stuff; giove it a few secs to "settle"
<spm> and responding to nagios checks; go for it
<lifeless> ok, I just got a request through.. turn it off again
<lifeless> then we'll grab the profile file and oops and I can nod off and examine it
<spm> haha. what a guess. that directory - completely unlooked for; already existed.
<spm> [profiling]
<spm> profile_requests: False
<spm> profile_dir: /srv/staging.launchpad.net/staging-logs/profiling
<spm> syncing logs...
<lifeless> bzr+ssh://bazaar.launchpad.net/~lifeless/lp-production-configs/timeouts has it in it now, but don't merge it - see special rollout notes
<spm> is restarted
<lifeless> so in that directory
<lifeless> there should be a bunch of files
<lifeless> and
<lifeless> possibly some oops's there, or otherwise they will be in the regular oops dir
<spm> sodium:/srv/launchpad.net-logs/staging/asuka/profiling/
<lifeless> spm: you can delete all the 2008 files ;)
<lifeless> hmm, but I can't see the participation one I would expect. time to grep em al I think
<spm> I was thinking that :-)
<lifeless> wgrant: https://bugs.edge.launchpad.net/ubuntu/+bug/602360 is a bug you might find interesting - soyuz query
<_mup_> Bug #602360: timeout on source package bug filing page <timeout> <Launchpad Bugs:Triaged> <Ubuntu:Invalid> <https://launchpad.net/bugs/602360>
<lifeless> spm: ah, its one of ScopedCollection:CollectionResource
<wgrant> lifeless: That plan makes me cringe.
<wgrant> It can't possibly need all of those tables.
<lifeless> curtis said, in a similar bug
<lifeless> https://bugs.edge.launchpad.net/launchpad-answers/+bug/608037
<_mup_> Bug #608037: timeouts in Question:+edit <oops> <timeout> <Launchpad Answers:Triaged> <https://launchpad.net/bugs/608037>
<lifeless> spm: actually, can you please nuke the 2008* files ;)
<spm> sure; done on asuka, will rm from sodium
<spm> gone
<lifeless> thanks
<lifeless> spm: also, can you look for oopses +- 1 minute from when I said 'done'
<lifeless> spm: and grep them for the url .../participation
<lifeless> there should be 2 I think
<lifeless> I just need the OOPS ids.
<spm>  /participants ?
<lifeless> yes
<spm> 03796.S56 03826.S95 are the files. go up one dir on sodium from the logs and into 2010-08-03
 * spm regrets cat'ing those out...
<lifeless> :P
<spm> OOPS-1676S95 OOPS-1676S56
<lifeless> thanks!
<lifeless> sweet - https://lp-oops.canonical.com/oops.py/?oopsid=1676S95
<lifeless> 1.8 seconds sql, 4.6 seconds python
<lifeless> 430 sql requests
<lifeless> also
<lifeless> haha:  WHERE Person.name = %s AND Person.merged IS NULL ORDER BY person_sort_key(Person.displayname, Person.name)
<lifeless> thats a little odd
<lifeless> exact match, oh but lets order it please.
<spm> *430* SQL requests! eek!
<lifeless> iz potato programming
<lifeless> now, I need to find some folk with detailed knowledge of the storm glue
<spm> as in like Mr Potatohead? just more bits stuck together?
<lifeless> 2010-08-03_02:03:40.261-ScopedCollection:CollectionResource-OOPS-1676S95-Dummy-2.prof is the profile file if you're interested
<spm> I shall have a look
<thumper> lifeless: what's up?
 * thumper knows glue well
<lifeless> thumper: hiya
<lifeless> have a look at  https://lp-oops.canonical.com/oops.py/?oopsid=1676S95
<lifeless> its a slow API call
<thumper> lifeless: we could go to voice if it helps
<lifeless> that might be a good bootstrap exercise
<lifeless> sodium:/srv/launchpad.net-logs/staging/asuka/profiling/2010-08-03_02:03:40.261-ScopedCollection:CollectionResource-OOPS-1676S95-Dummy-2.prof is the kcachegrind profile
<lifeless> thumper: https://api.edge.launchpad.net/beta/~ubuntu-dev/participants - open it in your browser
<lifeless> thumper: http://www.amazon.com/Patterns-Enterprise-Application-Architecture-Martin/dp/0321127420/ref=sr_1_1?ie=UTF8&s=books&qid=1280802410&sr=8-1
<lifeless> bbiab, chores to run
<poolie> happy performance day!
 * thumper done for nwo
<thumper> now
<lifeless> thumper: so, another question - can you point me at an example of testing the API / making a browser request - outside of a doctest ?
<bebo_mz> can any one help me in this W: GPG error: http://ppa.launchpad.net lucid Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 449F83829320B41C
<lifeless> you might have better luck in #launchpad
<adeuring> good morning
<lifeless> good morning
<didrocks> jml: hey, how are you?
<adeuring> lifeless: could you please explain a bit what the security issue is in my MP for lp:~adeuring/launchpad/bug-39674-update-retricted-flag-of-private-bugattachments ?
<mrevell> Morning
<jkakar> If someone has time to review a trivial Storm branch, please check this out, it adds an is_empty method to SQLObjectResultSet: https://code.edge.launchpad.net/~jkakar/storm/sqlobject-is-empty/+merge/31565
<adeuring> what should we for a buildbot failure "xvfb-run: error: Xvfb failed to start"?
<lifeless> adeuring: hi
<adeuring> hi lifeless
<lifeless> adeuring: its the whole arc of work - I filed a dedicated bug about it
<adeuring> lifeless: so, do you mean the ProxiedLFA stuff?
<lifeless> yes
<lifeless> bug attachments are user controlled content
<lifeless> if they are served in a domain, they can run scripts to access anything else in the domain - other bugs, private lists, etc, and submit the content to other servers
<adeuring> lifeless: right, but I don#t see the issue yet... anyway, can you tell me the bug number?
<lifeless> its why the librarian is on launchpadlibrarian.net rather than part of the appservers
<lifeless> 612779
<lifeless> adeuring: the issue is, that anyone can file a private bug, with an attach script, which will (for instance) steal the users oauth cookie and forward it to a hostile site
<adeuring> lifeless: yes, but that applies to attachments of bugs too...
<lifeless> no it doesn't
<lifeless> because they are on a different domain
<adeuring> ah, right, forgot that
<bigjools> lifeless: see my comment on your librarian MP
<lifeless> bigjools: https://code.edge.launchpad.net/~lifeless/launchpad/private-librarian/+merge/31020 ?
<lifeless> bigjools: or the oops one ?
<lifeless> I can't see a comment on the private-librarian MP
<bigjools> lifeless: 31508
<lifeless> full url please ?
<lifeless> bigjools: I've already put it on the wiki page a week ago ;)
<bigjools> lifeless: teh cool
 * thumper fixes the merge conflict
<lifeless> \o/ https://bugs.edge.launchpad.net/ubuntu/+filebug is live with my tweaks
<adeuring> lifeless: ok, I get it now. So, any estimate when you can land your librarian branch?
<lifeless> adeuring: well my branch doesn't really help
<lifeless> it has the same concern
<lifeless> adeuring: I think what we can sensibly do for now is set cd:attachment on untrusted restricted objects.
<lifeless> adeuring: this may require a new field on LFA, *or* a new view for how they are traversed.
<adeuring> lifeless: right, that's the other option. I just wondered if this is worth the effort since your work would fix the problem as well
 * thumper fixes stable -> db-devel conflicts
<lifeless> adeuring: so my work will move the private content off the launchpad main domain
<lifeless> adeuring: but it will still let private content attack other private content
<adeuring> lifeless: right
<lifeless> adeuring: so I think we should *also* take steps to make it less risky
<adeuring> lifeless: makes sense. A variant of ProxiedLFA looks reasonable
<stub> lifeless: Why doesn't your branch help? I thought with your work, restricted files would be served on the public port on its seperate domain.
<stub> oic
<lifeless> stub: it helps, but its not the full story
<stub> But there is nothing to attack, as no cookies are set.
<lifeless> stub: sadly there is
<stub> stealing URL history or something?
<stub> I guess we could use a wildcard and serve files from https://abctoken.launchpadlibrarian.net/123/foo.txt instead of https://launchpadlibrarian.net/123/foo.txt?token=abctoken
<lifeless> stub: window A can access the DOM of window B
<lifeless> thus the urls (and tokens) for other private content, and then submit that url encoded to $attckingstite
<lifeless> stub: yes, wildcard dns is what I'm thinking will work.
<lifeless> (Window A and window B have to be on the same domain for the above attack to work)
<lifeless> stub: I was thinking https://contenthash.launchpadlibrarian.net/foo.txt?token=abctoken or something like that
<stub> If you use the LFA.id, https://f123.launchpadlibrarian.net/foo.txt instead of the existing https://launchpadlibrarian.net/123/foo.txt, and allows us to rewrite existing URLs.
<lifeless> stub: its rather late for me, but I'm totally keen on whatever permutation will work best for us
<lifeless> stub: so if you'd like to shove that into the MP discussion on https://code.edge.launchpad.net/~lifeless/launchpad/private-librarian/+merge/31020 with any hints about what glue I'll need to piece it together, that would be cool.
<stub> I think you want spiv or jml or someone who has worked with twisted http for the hints. I can add this discussion though.
<lifeless> the twisted server internals I am fine on
<lifeless> its more things like the dns setup, apache/squid changes etc - just a bookmark of the gotchas
<lifeless> also, yay, bug filing searches seem solidly < 10 seconds to me now, on edge.
<bigjools> lifeless and stub: can you bless the db patch and give me a number please: https://code.edge.launchpad.net/~julian-edwards/launchpad/buildd-failure-counting/+merge/31556
<lifeless> what TZ is deryck in ?
<lifeless> ah, us.
<jkakar> bigjools: Are you pumped?!  Ready to have a great time!!!  https://code.edge.launchpad.net/~jkakar/storm/sqlobject-is-empty/+merge/31565
<bigjools> jkakar: \o/
<bigjools> jkakar: I think we should also fix the bug about removing the ordering on .any()
<bigjools> lifeless is on a speed war
<lifeless> \o/ phwoar
<jkakar> bigjools: It's fixed in another branch and al-maisan had agreed to review it!
<jkakar> Once that's done it'll land, hopefully today.
 * al-maisan just approved that branch
<bigjools> \o/
<bigjools> jkakar: when will it be released?
<lifeless> mpt: hai
<mpt> oh hai
<jkakar> bigjools: As a package, you mean?
<jkakar> al-maisan: Thanks!
<lifeless> mpt: I'm all excited - bugs.edge.launchpad.net/ubuntu/+filebug
<lifeless> mpt: try it. Marvel at the speed.
<lifeless> [actually, it still has lots of room to improve, but I'm easily pleased]
<bigjools> jkakar: we use eggs for buildout
<jkakar> bigjools: So you need a tarball, right?
<bigjools> I guess so, I'm not that familiar with that stuff
<jkakar> bigjools: Or I guess an egg, but we don't create those when we make releases as far as I know.
<jkakar> bigjools: What would it take to start using the python-storm package?
<mpt> lifeless, "lifeless is excited at the speed" returns suggestions in <2 seconds. "Ubuntu sucks" returns suggestions in ~13 seconds.
<mpt> Pretty sure that's an improvement. :-)
<jkakar> Also, fwiw, we use buildout and have it checkout lp:storm which works quite well, since we're always on trunk.
<lifeless> mpt: yes, sadly 2 term searches will still be slow until we get a new search layer.
<bigjools> jkakar: no idea!  I don't know the pros/cons of that versus what we do, someone from Foundations would know though.
<bigjools> jkakar: interesting, I'll get gary to talk to you
<jkakar> bigjools: Get him to talk to sidnei. :)
<lifeless> mpt: as in, 2-term searches will be the slowest, if you graph speed vs term count
<bigjools> jkakar: o7
<bigjools> wgrant: howdy.  I don't suppose you've QA verified any of your fixes this cycle?
<lifeless> stub: so, is a 'dba' bug tag ok with you?
<stub> Sure, but I need to work out how to get GMail to flag them. I seem to require push methodologies :)
<wgrant> bigjools: My stuff is just about impossible for me to QA.
<bigjools> I figured :(
<wgrant> Sorry.
<lifeless> stub: ok, well I'll start marking up :)
<lifeless> stub: would an rss feed work ?
<stub> I always forget to check my RSS feeds :)
<lifeless> hah, ok
<lifeless> [me too]
<lifeless> deryck: oh hai
<deryck> lifeless, hey, man.
<wgrant> So, I have a whole lot branches that need to be reviewed, just about no reviewers in my timezone, there's a lack of European OCRs, and American OCRs tend to knock them off the queue if you're not around.
<jelmer> wgrant: I'm OCR tomorrow, should have some time to review them then.
<wgrant> jelmer: Thanks.
<lifeless> wgrant: also, I can review them all tomorrow.
<lifeless> wgrant: I wasn't today because performance day!
<jelmer> hmm, actually
<jelmer> jtv: are you OCR as well tomorrow? Perhaps I should do Thursday this week as noodles is on holidays.
<jtv> jelmer: yes, I'm on tomorrow
<jtv> jelmer: I'll finish around mid-day for you though
<jtv> Who removed installKarmaRecorder from TestCase?
<jtv> Oh crap... the entire KarmaRecorder is gone.
<jml> jtv, https://code.launchpad.net/~henninge/launchpad/bug-612583-test-errors/+merge/31546 is the only mention of KarmaRecorder in my mailbox
<jtv> jml: we must not have been using it for a while.
<jtv> jml: hmm... that mp says it was introduced in the recife branch.  I don't remember that; I thought it was older.  Thanks.
<jml> jtv, np
<wgrant> lifeless: Thanks.
<wgrant> They're mostly small.
 * lifeless -> sleep
<didrocks> jml: hey, how are you?
<jml> didrocks, hello, I'm well thanks.
<didrocks> jml: I didn't have the time to talk to you before because of GUADEC last week and didn't see you at the sprint apart from the first day, did you have some time to have a look at the gpg/ssh/ppa api?
<jml> didrocks, no, sorry. it's on my list though. I think the next step is creating a new permission level
<jml> which maybe I can convince someone else to do
<didrocks> jml: let's hope so, no hurry for now. I'll ask for a feature freeze exception on the Quickly side. Thanks :)
<jelmer> mars: Hi
<jelmer> mars: I've build a new EC2 image and would like to bless that as the new official image. As far as I can tell I've done everything mentioned on the wiki, but "ec2 test" locally is still using the old version.
<jelmer> mars: I've added myself to the list of valid AMI owners, and made my image public.
<jelmer> mars: Is there anything I might have forgotten?
<bigjools> jml: thanks for the extra twisted tip BTW, that's sneaky
<jml> bigjools: np. as I said, perfectly valid in tests and in some extremely rare real code cases.
<mars> jelmer, gary_poster might know if the image is ready to go
<mars> I haven't actually built one myself yet, but gary_poster and mwhudson have
<mars> jelmer, ack, gary_poster is not around right now, but he should be back before your EOD I would suspect
<gary_poster> I am, I am :-)
<jelmer> mars: Ok, thanks!
<jelmer> ah! Hi Gary
<gary_poster> jelmer, hi. public image and valid AMI owner: sounds right to me
<gary_poster> do you want me to see if I can see your image on the management console or something?
<jelmer> gary_poster: It'd be great if you can check, although it is marked as public here.
<gary_poster> k
<jelmer> ami-217f9448 is the ID
<jelmer> gary_poster: Sorry, unping. I've found the issue.
<gary_poster> jelmer: cool.  fwiw, I couldn't find it
<gary_poster> I mean, the ami
<jelmer> the branch I was submitting didn't have myself added to the list of valid AMI owner IDs yet.
<gary_poster> ah :-)
<jelmer> gary_poster: sorry for the trouble, thanks for thinking along :-)
<gary_poster> np :-)
<james_w> could someone tell me what a reasonable check for calling removeSecurityProxy is?
<gary_poster> james_w: "reasonable check"?
<james_w> exactly
<james_w> UnreasonableRemoveSecurityProxyWarning: Called removeSecurityProxy(<PackageUpload at 0xda8906c>) without a check if this is reasonable.
<gary_poster> sorry, was intending to convey the idea that I didn't understand that part of the question
<gary_poster> oh
<james_w> neither do I
<gary_poster> abel knows about that
<gary_poster> he isn't around
<gary_poster> deryck[lunch] knows about that...
<gary_poster> of course, I'd grep for the error and see if the code path explained the intent
<gary_poster> I'm supposed to be doing several other things so I'll leave that to you, I'm afraid
<james_w> it appears to be rather than fixing the fallout caused by returning proxied objects
 * james_w replaces it with removeSecurityProxy, I can't see any reason why that would be wrong
<jml> james_w, hey
<james_w> hi jml
<jml> james_w, what that error means is that adeuring changed the code to use rSP but isn't sure whether it was appropriate
<jml> james_w, so if it is appropriate to use rSP (which IMO basically it is in all cases in a test except when you're testing security) then change the call to removeSecurityProxy
<james_w> right, this is in SoyuzTestPublisher, so if there is application code using it then we have bigger issues
<jml> the alternative is to log in as an appropriate user
<adeuring> jml, james_w: yes's what I meant: I simply added these calls mecahnically ust to ensure that tests do not fail due to added security proxies
<jml> adeuring, thanks.
<jml> maybe there's a better way of phrasing the warning...
<rockstar> bigjools, is there an open bug about getting new build hardware?  I'd love to mark this bug as a dupe: https://bugs.edge.launchpad.net/launchpad-code/+bug/612177
<_mup_> Bug #612177: Built daily? Really? <recipe> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/612177>
<bigjools> rockstar: not on soyuz, no.
<rockstar> bigjools, should there be?
<bigjools> rockstar: it's not really a bug
<bigjools> it's an RT
<bigjools> however, we are working on some performance improvements but given the number of jobs versus number of builders, we're never going to win
<bigjools> rockstar: search for the "buildfarm" and "buildd-manager" tags and see if any fit for a dupe
 * rockstar waits forever for dogfood...
<jml> abentley, if I wanted to fix bug 608114, would I need to change anything other than Diff.fromTrees in lp.code.model.diff?
<_mup_> Bug #608114: Show modified class/method names in the diff of the merge proposals <code-review> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/608114>
<abentley> jml, that's a request to stop using bzr to perform diffs and start using /usr/bin/diff.
<jml> abentley, does bzr switch to using external diff only when external diff options are provided?
<abentley> jml, yes.
<jml> abentley, is there any reason we shouldn't be using external diff?
<abentley> jml, bzr produces better diffs.
<jml> abentley, I see.
<abentley> jml, and bzr produces different diffs.  So people will see differences between the bzr output they produces locally and the diffs that Launchpad generates.
<jml> abentley, I don't think last one is necessarily bad.
<jml> abentley, producing worse diffs is enough to make me not want to go for the easy fix for the bug, though.
<abentley> jml, variation breeds confusion.
<jml> among other things
<jml> I wish more things were easier.
<benji> words to live by
<jml> abentley, ok, thanks. I've commented on the bug, including a patch that fixes it in the bad way
<jml> abentley, I just saw https://bugs.edge.launchpad.net/testrepository/+bug/611706 -- do you think something like the following would be readable?
<_mup_> Bug #611706: Confusing output <Testrepository:New> <https://launchpad.net/bugs/611706>
<jml> id=0, tests=21, failures=4 skips=1
<jml> err... imagine there's a comma after the '4'
<abentley> jml, yes, I think that would help me read it properly.
<jml> abentley, thanks. I'll submit a patch.
<jml> james_w, are you still being bitten bug 597060?
<_mup_> Bug #597060: testr run should support limiting ids <testr-run> <Testrepository:New> <https://launchpad.net/bugs/597060>
<james_w> jml: yes, in that I've learnt not to do it and so don't use testr as much
<jml> hmm :(
<jml> james_w: maybe I don't understand it then.
<james_w> Rob wants to solve it by working on the bug called something like "$IDOPTION needs some thought"
<james_w> jml: I just want to be able to do the parallel of "test run lp.code". With $IDLIST this runs the everything, and then the code tests the second time.
<james_w> (not that it does that on LP, as LP uses IDOPTION, so I don't know what happens)
<jml> ahh ok
<jml> james_w, the source of my confusion is that I was unaware of how flexible .testr.conf files really are
<mars> jml, quick silly question: how do I run tribunal from a fresh checkout?  The README doesn't say how.
<jml> mars, ./bin/tribunal
<jml> mars, ./bin/tribunal-subunit, actually
<mars> hmm, no bin/ in the lp:tribunal checkout
<jml> mars, my local copy appears to be out of date
<mars> usefully out of date :)
<jml> mars, ./tribunal-subunit, then
<mars> jml, ok, thanks.
<jml> (as an aside, Launchpad is a bit weird in that it's the only project I know that has binaries in bin/ that aren't intended for distribution)
<mars> I run OpenKomodo mainline, and they have a project-specific bin/ for developers
<mars> The project is built from Python and JS wrappers over Mozilla XUL, so it's a fairly large project, too
<mars> ok, so "~/src/tribunal/tribunal-subunit ." works
<mars> it picks up the .testrepository dir automagically
<lifeless> moin
<jml> lifeless, hi
<lifeless> hi
<jml> lifeless, I'd love to talk with you about all the stuff I just did to testrepository, but I also want to escape my computer.
<lifeless> well
<lifeless> you could ring me from your noncomputer
<jml> nah, same thing. thanks though.
<jml> lifeless, none of it's urgent, so I think I'll just sign off.
<jml> have a good day
 * jml off
<lifeless> ok, ciao
* lifeless changed the topic of #launchpad-dev to: Launchpad Development Channel  | Week 3  of 10.08 | PQM will be closing 22:00 UTC Friday | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<james_w> I've just found two objects that don't correctly implement their interface, I wonder how many more there are
<lifeless> N
<lifeless> :P
<wgrant> james_w: Lots of classes have verifyObject called on them.
<wgrant> But I guess some do not.
<wgrant> mars: Thanks for the reviews.
<wgrant> mars: yes, lots of those doctests should be unit tests instead, but that basically requires a rewrite of a large portion of the Soyuz test suite.
<thumper> morning
<wgrant> That's going to take a while.
<james_w> a "fun" task is going to be signing an upload during the tests so that an ubuntu series isn't hardcoded in the test data
<james_w> though perhaps the best way to do that is to wait until that day that we remove the sampledata and use the factory to create a series with a matching name
<wgrant> That was going to be my suggestion.
<mars> wgrant, yep, I know.  Julian and I had a discussion about that very problem earlier today.
<wgrant> mars: I dissect them where I can, but there's a lot left...
<thumper> db-devel branch failing with testMemcachedWorking
<thumper> anyone know of recent changes in that area?
<mars> thumper, yep
<mars> looking
<thumper> ta
<mars> Hurray for test suite instability!
<mars> thumper, losas to fix both, I'll tell them.
<thumper> mars: what do you mean?
<mars> thumper, both builders are dead on test infrastructure failures
<thumper> mars: jelmer is putting in an RT for the devel failure, old ec2 image I'm told
<thumper> mars: not sure about the db-devel failure
<mars> thumper, ah, that might do it
<thumper> mars: jelmer was going to roll-back the last commit (that needs the new package)
<mars> yep
<mars> might not be that simple
<mars> Tom updated that machine to Lucid, which means it's off EC2 now.  If that is true, then a new image won't help
<mars> lifeless, ping, did the --load-list option for the zope testrunner get landed or released upstream?
<mwhudson>  morning
<lifeless> mars: I don't know
<mars> ok
#launchpad-dev 2010-08-04
<wgrant> mars: I don't understand your assertThat suggestion.
<wgrant> I've not seen it before.
<mars> wgrant, it's something newish that testtools ported from Java.  You write a custom matcher and pass it into the assertThat() method along with the object to be asserted
<mars> wgrant, so you would write a matcher that does the "assertEqual(expected, list(object._myMethod()))" dance for you, as if you were writing a helper function or something
<mars> then you pass that matcher and the object under test into assertThat:  assertThat(myobject, PassesMyTest(foo))
<wgrant> Ah, I see.
<mars> The Java guys came up with it as a hack because assertEquals in Java just prints "True" or "False", which is really lame.  So they created the assertThat() method so that it can actually print nice test failure messages.
<mars> However, it also serves as a nice way to pull common custom test conditions together as readable wrapper for helper functions
<mars> wgrant, here's the code: http://bazaar.launchpad.net/~testtools-dev/testtools/trunk/annotate/head:/testtools/matchers.py
<wgrant> I see it's not used anywhere in LP.
<wgrant> I'm not sure how to write a matcher.
<mars> wgrant, ok, maybe don't worry about it.  I'm just reading the code now, and you have to define a new class to do it.
<wgrant> mars: Do you object strongly enough to NascentUpload's conditional __init__ that I should change the several dozen callsites which pass in a path?
<lifeless> wgrant: give me a few minutes and I can help; or you can look at at testtools.matchers, bzrlib.test.matchers, or james' external matcher project
<wgrant> I suppose it's nicer.
<poolie> hi mars, wgrant
<wgrant> Morning poolie.
<mars> lifeless, I was thinking it was a nice way to wrap a repeated one-line assertEquals(), but I don't think the matchers are light enough to do that yet
<mars> Good morning poolie
<poolie> i've been meaning to try joining up multiple related assertions under an aggregating matcher
<poolie> so you get a better message when they fail
<mars> A decorator to somehow turn a helper function into a matcher would do the trick, but it's just a fancy way of writing a self.checkThat(foo) method.
<lifeless> mars: wgrant: whats the MP that this is being discussed on ?
<poolie> i think there's a social effect here that we don't normally have tests failing very much
<wgrant> lifeless: https://code.edge.launchpad.net/~wgrant/launchpad/test-ddeb-matching/+merge/31482
<poolie> so having nicer failures is only a transient benefit
<mars> poolie, here as in Launchpad, or here as in Python?
<poolie> in bzr, and probably in lp
<poolie> maybe you get more failures in buildbot in lp
<lifeless> wgrant: ok, matchers will definitely be cleaner. let me sketch you a story
<wgrant> lifeless: Cleaner than a domain-specific assertion method?
<poolie> yeah, they would be
<lifeless> yes
<wgrant> Hm.
<poolie> definitely cleaner than inventing a per-test way of factoring it out
<lifeless> wgrant: you also have some dead code AFAICT
<wgrant> Hm, there's an 'except KeyError' that probably shouldn't be there any more.
<wgrant> But what else?
<wgrant> Oh, no, that should be.
<wgrant> What's dead?
<lifeless> wgrant: http://paste.ubuntu.com/472863/
<lifeless> wgrant: I didn't quite get the intent of your function initially
<lifeless> wgrant: thats a sketch of a matcher : use it as self.assertThat(self.changes, HasCorrectDebugDebs())
<lifeless> it has the following properties; it is itself testable
<wgrant> lifeless: Um, matchDDEBs is production code.
<wgrant> Not just for tests.
<lifeless> wgrant: thats interesting :) this can still fit in that context - matchers are not test-only code.
<lifeless> anyhow, if it fits - great; if it doesn't, thats fine too.
<mars> lifeless, poolie, in your 'switchable sandbox' workflow, does 'bzr push' still work as normal from within the tree'd branch you are working in?
<james_w> wgrant, lifeless, mars: there will be lp.testing.matchers landing very shortly
<mars> or do I have have to cd out and into the branch I am switched to?
<james_w> I would encourage lp.<app>.testing.matchers or similar too
<mars> james_w, cool, be sure to write the list when it does
<mars> very nice
<james_w> I think it's either in PQM's queue, or got lost between ec2 and pqm
<thumper> :(
<thumper> I want to update the pre-requisite branch of a merge proposal
 * thumper tweaks todo list
 * mwhudson waves at james_w 
<lifeless> Ursinha-afk: doh, I misse dyou
<james_w> hi mwhudson
<mwhudson> james_w: i was going to ask how things were going, but maybe we should do that in #linaro
<wgrant> How is stable ever going to get deployed in the new QA model?
<wgrant> If it has to be fully QAd...
<james_w> where do I find launchpad pqm's web interface?
<mars> james_w, https://pqm.launchpad.net, but it won't help you much
<mars> james_w, you might have to grep the server logs to find your branch
<mars> oh, or not
<mars> the branch is in the queue
<lifeless> wgrant: a specific rev of stable is fully qa'd, not tip.
<lifeless> wgrant: I'll make that clearer
<wgrant> lifeless: Ah, I see.
<lifeless> mars: yes, it still works.
<wgrant> lifeless: flacoste's email this morning confused me.
<lifeless> yes
<wgrant> >      Is there a facility to roll out just the QAd stuff?
<wgrant> No. Un-qaed revisions blocks deployment.
<lifeless> what he means is
<wgrant> It'll roll out an earlier revision.
<lifeless> we can only rollout qa'd stuff; and that means we can't skip a revision
<lifeless> if you have ABCDEF and A,C are QA'd we can only deploy A
<lifeless> B needs to be either rolled back, or QA-ok.
<wgrant> Right.
<lifeless> anyone seen ImportError: No module named debian on ec2 recently ?
<wgrant> That rev was just rolled back.
<wgrant> But I thought buildbot was broken -- not ec2.
<mars> wgrant, he may have fired up ec2 before the fix landed
<jelmer> lifeless: You need to merge in a more recent lp:launchpad/devel, which adds me to the list of valid image owners. Though it's not relevant anymore since that revision has indeed been reverted.
<lifeless> jelmer: into where - the ec2land running branch, my branch being submitted, or the target ?
<jelmer> lifeless, the ec2land running branch
<lifeless> ugh
<lifeless> but ok
<lifeless> is there a bug about the hot bugs list not updating bug status ?
<poolie> lifeless: yes, is memcache misuse bug
<lifeless> poolie: yes, but whats the bug #
<wgrant> The milestone view is still broken too :(
 * poolie looks
<poolie> ah bug 599614 is the milestone page
<_mup_> Bug #599614: bad caching in milestone page <qa-ok> <trivial> <Launchpad Registry:Fix Released by sinzui> <https://launchpad.net/bugs/599614>
<wgrant> It's a lie.
<wgrant> Ah, different bug.
<poolie> lifeless: i wonder if we should 1- just turn them all off; 2- turn them right down to be unobjectionably low; 3- push expiry; 4- force browser refresh
<wgrant> Bug #601051 is one.
<_mup_> Bug #601051: Caching of primary data sources reduces utility <Launchpad Registry:Triaged> <https://launchpad.net/bugs/601051>
<poolie> so this is an interesting case to use approximation
<poolie> if you say "about 200 open bugs" it's a bit more tolerant of being out of date,
<poolie> but not really, because i could speedily close many of them, and then it would still be wrong
<poolie> wgrant: that's a nice description of a general cause
<poolie> is it really a registry bug?
<wgrant> Well, the problematic cases are mostly Registry.
<wgrant> But possibly not.
<poolie> bug 604530 too
<_mup_> Bug #604530: brower refresh button should refresh TAL caches <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/604530>
<james_w> is http://paste.ubuntu.com/472894/ not broken?
<wgrant> I'd say so.
<wgrant> It's been BPPH for yeeeears.
 * mwhudson hasn't internalized the soyuz model well enough to say off hand
<james_w> there's no distroarchseries on BinaryPackageRelease or its interface
 * thumper has connection issues
<james_w> make that three classes that don't implement the interface
<thumper> there are people outside messing with wires
<wgrant> james_w: The idea is right. It's joining BPR to DAS via BPP.
<wgrant> Except that BPP is BPPH and has been for a long time.
<james_w> does this imply that the "packages" attribute isn't used anywhere?
<wgrant> I would hope that was the case.
<mwhudson> james_w: is it in the interface?
<james_w> mwhudson: yes
<mwhudson> sigh
<james_w> delete it or write tests for it?
<james_w> I'm inclined to delete it
<mwhudson> delete it
<mwhudson> if it's not used
<mwhudson> which it seems like it can't possibly be
<lifeless> thumper: telecomnz seems to have had an outage
<spm> someone let that wet string dry out again?
<lifeless> :P
<james_w> the first set of changes to wean SoyuzTestPublisher off the sampledata is now in ec2 to generate a list of fallout for me to work through
<wgrant> That could be amusing.
<james_w> I'm doing it piecemeal
<wgrant> I'm considering demolishing parts of builder.txt.
<james_w> this doesn't just make it ignore sampledata. There's one pipe to make the SoyuzTestPublisher back on to the factory, and another to give an API to avoid sampledata, with minimal changes to existing test paths.
<wgrant> Soyuz tests seem to be getting a bit of a makeover at the moment.
<wgrant> Ah, excellent!
<james_w> that will allow us to gradually migrate
<james_w> if it turns out there isn't much fallout then I'll just switch the existing code paths. The branch I just sent to ec2 does this for all the tests in test_publisher, just to get a feel for the fallout.
<lifeless> wgrant: so, I think I've captured what you wanted in bug https://bugs.launchpad.net/launchpad/+bug/601051
<_mup_> Bug #601051: memcache is used as a bandaid for responsiveness <Launchpad itself:Triaged> <https://launchpad.net/bugs/601051>
<lifeless> wgrant: if instead you meant to say 'page X is cached and shouldn't be', please change it to say that and put it on the right product.
<wgrant> lifeless: Indeed, thanks.
<wgrant> There are a few around now, so your changes sound good.
<thumper> OMFG, my code is actually working
<spm> thumper: that's the beauty of statistics; eventually that million to one chance comes up!
<thumper> haha
<spm> (I am *so* dead at the next allhands...)
<thumper> fcuking aussies think they are better than the rest of us
<spm> s/think/know/ ;-)
<thumper> I was amused to read an article on the sydney morning herald website
<thumper> about how australia was going to beat the all blacks last weekend
<thumper> one of his 10 reasons was "because australians are better than kiwis at everything"
<spm> ah yes. The SMH. Purveyor of Kwality Journalism.
<wgrant> Excellent reasoning.
<thumper> another was "because it is about time"
<thumper> and "we have too good a coach to lose to the all blacks 8 TIMES IN A ROW"
<spm> I gather the AllBlacks won then?
<thumper> heh, by lots
<spm> ha! nice.
<thumper> 49 to something small like 14
<spm> orsum!
<spm> was it a decent match depsite the score diff? or just embarrassing?
<thumper> wallabies got someone red carded for the first time ever against the ABs
<spm> Ahh. Biased ref eh?
<thumper> no
<thumper> stupid player
<spm> (zing. trolled!)
<thumper> ABs had a player yellow carded for not using arms in a tackle (read as shoulder charge)
<lifeless> oof
<thumper> Wallabies got a player yellowed for exactly the same reason with 5 minutes
<thumper> ref warned both captains to stop slowing the ball down and he would move to yellow cards for anyone who continues
<lifeless> I wonder
<lifeless> for tag clouds
<spm> hrm. ok, It's been about 25 years since I last played rugby; but we were taught to more or less hit with the shoulder and wrap the arms around to pull 'em down. that no longer the case?
<lifeless> if using just the last 10K modified or something to summarise would be good
<lifeless> rather than 'all open'
<thumper> wallaby player gets yellow carded for slowing the ball down, but it was the same guy, 2 yellows = red
<lifeless> (which we don't even do today)
<thumper> spm: that's kinda OK as long as the arms are up and out at the time of shoulder contact :)
<spm> lifeless: I'd suspect it would be  - if only for 'most recent ~= most relevant'
<wgrant> lifeless: Don't we use 'all open' at the moment?
<wgrant> lifeless: Apart from the dupe bug.
<spm> thumper: roight
<lifeless> wgrant: bug 404*** - tags from closed bug sshow in the cloud
<lifeless> wgrant: that might be dupe related, I dunno
<wgrant> Hmm.
<lifeless> wgrant: check your bug mail ~ 2am
 * thumper is making bzr push lp:project work when there isn't yet a trunk
<lifeless> thumper: orsum
<wgrant> lifeless: Ah.
<lifeless> losa ping
<spm> yo
<lifeless> whats the unit on https://lpstats.canonical.com/graphs/MemcachedServersAll/
<lifeless> requests per ??? ?
<spm> tribble
 * lifeless trouts spm
<thumper> spm: there is something I'm supposed to be talking to you about...
 * thumper thinks
<spm> lifeless: I think it's seconds tbh.
<lifeless> spm: and https://lpstats.canonical.com/graphs/MemcachedTotal/
<lifeless> spm: does that mean, as I think it does, that we're getting 20% hit rate?
<spm> lifeless: 5 min average, or possibly 1 min avg; per second.
<spm> lifeless: I guess so - I'd suggest verifying with stub tho; he's more across the fine details of those than me
<lifeless> meh, I'll shoot my foot off, its fine.
<spm> fwiw, the serversall is a tad busy, but somewhat by (my) choice. was thinking that if we note server X is showing numbers wildly different to A, B & C, then there is "A Problemâ¢"
<spm> but by and large, they all track fairly close together, which is a good thing
<thumper> mwhudson: please tell me that if the xmlrpc server returns a fault it aborts the transaction
<mwhudson> thumper: i think so, you should be able to confirm by looking at the publication code
<thumper> mwhudson: where is the xmlrpc publication code?
<thumper> do you know?
<mwhudson> thumper: otp
<thumper> ack
<thumper> :(
<thumper> lp.codehosting.inmemory is going to make my life hard
 * lifeless sets up a flamefest on lp-dev
<thumper> lifeless: about?
<lifeless> memcached
<mwhudson> is the set up you need to pqm-submit launchpad branches documented somewhere?
<mwhudson> i know it used to be on launchpad.canonical.com
<lifeless> if this is wgrant's stuff, use ec2land ;)
<mwhudson> it's just generally trying to recover my set up
<mwhudson> i didn't lose much data, but i hadn't backed up lots of configuration :(
<lifeless> here is what I used:
<lifeless> echo "star-merge bzr+ssh://bazaar.launchpad.net/~lifeless/launchpad/malone  bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel" | gnome-gpg --cl | mail launchpad@pqm.canonical.com -s "[r=mwhudson,thumper][ui=none][bug=607776] Further trade off ideal dup detection for speed, with an eye to bring back in ideal detection soon."
<lifeless> when I had something that I needed to do directly :P
<thumper> I just use the pqm-submit plugin, and have an alias for db-submit
<thumper> mwhudson: you need (or should have) the canonical smtp server set up for outgoing email
<thumper> mwhudson: [/home/tim/repo/canonical/launchpad]
<thumper> pqm_email = Launchpad PQM <launchpad@pqm.canonical.com>
<thumper> submit_branch = /home/tim/repo/canonical/launchpad/devel
<thumper> mwhudson: and set a public location for the submit branch
<mwhudson> thumper: ta
<lifeless> hmm
<lifeless> my memcached mail seems to have gone awol ?
<lifeless> either that or lp list handling has gone boom ?
<lifeless> spm: ^?
<spm> I've got your email. ?
<spm> "So, we've got a partial deployment of memcache in place..."
<lifeless> ok cool. just gmail being stupid I guess
<lifeless> not showing me what I sent ...
<mwhudson> thumper: the publication code is in canonical.launchpad.webapp.servers
<james_w> mwhudson: please add to https://dev.launchpad.net/LandingChanges
<mwhudson> thumper: and yeah, inmemory doesn't support transactions
<mwhudson> james_w: actually maybe i am just being very antiquated in wanting to use pqm-submit
<wgrant> lifeless: One concern: as of 10.08 the front page makes requests to the blog to retrieve the feed. That's stored in memcached.
<wgrant> I'm not sure if that's going to be a problem, though.
<lifeless> well
<lifeless> we can keep that one page I guess.
<lifeless> (that said, the blog is local in the DC isn't it ?)
<wgrant> I think so.
<lifeless> remember that right now memcache isn't 'production available' - its not rated as a sev 1 when its down.
<lifeless> poolie: you said the other day that we could talk feature flags
<lifeless> poolie: did we? I forget
<poolie> hey
<poolie> i did say that; we didn't talk; talking now could be good
<poolie> this is a good time for me actually
<poolie> skype?
<lifeless> sure
<lifeless> poolie: https://bugs.edge.launchpad.net/ubuntu/+filebug-show-similar?title=lifeless%20is%20excited%20by%20the%20speed
<j24> hi
<j24> is there any doc howto install and configure launchpad dev?
<poolie> j24: dev.launchpad.net
<j24> I've read it but not clear
<j24> I've install it
<j24> poolie: I've install by apt-get
<j24> and now want to run it
<j24> but it never start
<poolie> what did you install?
<j24> launchpad-integration
<j24> $ co
<j24> & co
<poolie> and what are you trying to accomplish
<j24> I want a local launchpad
<j24> to manage our own projects
<poolie> ok
<poolie> the "launchpad-integration" package isn't the launchpad source
<j24> I've install this
<poolie> https://dev.launchpad.net/Running
<j24> http://paste.ubuntu.com/472926/
<poolie> yeah, that's not actually launchpad
<poolie> as the name says, they're the dependencies and integration code
<j24> how can I install it so?
<j24> I want to install how own launchpad server
<poolie> read that wiki page
<j24> is there any package that can do it on ubuntu?
<j24> with have to install the source
<j24> I've read the wiki but they always talk about manual settings
<j24> not howto install your own launchpad
<j24> poolie: if I download the source ok
<j24> it work if I run make run
<j24> but to setup a server it not good
<j24> poolie: how can people will use bazar for them self when it's not friendly at all
<j24> and ditto for launchpad
<wgrant> j24: Launchpad's not designed for you to install it yourself.
<wgrant> What is the prolbem that you have with Bazaar?
 * mwhudson can't remember if he used to use gnome-gpg or seahorse
<thumper> j24: I use bzr locally without launchpad for most of my work, what is it you are trying to do?
<thumper> StevenK: ping
<lifeless> spm: yo! OSError: [Errno 2] No such file or directory: '/var/lock/launchpad-codeimportdispatcher.lock'
<StevenK> thumper: Pong
<cody-somerville> Is this how adapters are intended to be used?: http://www.muthukadan.net/docs/zca.html#adapters
<thumper> cody-somerville: not necessarily how we do it
<thumper> cody-somerville: you might want to take a look at BranchTarget as an example of how lp.code does it
<cody-somerville> Does launchpad use invariants, handlers, or subscription adapters?
<mwhudson> i don't know what a subscription adapter is
<mwhudson> i don't think we use invariants
<mwhudson> what do you mean by handlers?
<mwhudson> we do use zope.event a bit
<spm> lifeless: huh. so I see. two of 'em on diff servers. curious....
<cody-somerville> mwhudson, handlers are subscription adapter factories that don't produce anything.
<cody-somerville> mwhudson, http://www.muthukadan.net/docs/zca.html#subscription-adapter
<lifeless> cody-somerville: no, we use a somewhat simpler thing
<lifeless> notify(event)
<lifeless> even implements some IFoo
<lifeless> and things that have register(IFoo) get called back
<mwhudson> cody-somerville: ok, no i don't think we use subscription adapters
<lifeless> they look like a solution looking for a problem, to me ;)
<wgrant> We probably use them for publication hooks, but that's all I know of.
<StevenK> lifeless: O hai, no review?
<lifeless> grah
<lifeless> url me up
<lifeless> I shall do
<StevenK> lifeless: https://code.edge.launchpad.net/~stevenk/launchpad/move-ifp-from-idistroseries/+merge/31520
<lifeless> StevenK: so
<lifeless> INSERT INTO is one case
<StevenK> I've since converted these calls to store.execute(), by the by
<lifeless> StevenK: I'm looking at store.execute on the merge page :P
 * StevenK kicks his browser
<lifeless> _copy_lucille_config can be done with the GenericCollection stuff
<lifeless> its a set-wide mutation, which it supports.
<lifeless> right
<lifeless> so - INSERT INTO needs a storm feature, please file one (it needs INSERT INTO support, which AFAIK is not yet available)
<lifeless> the UPDATE can be done already without writing SQL.
<StevenK> I'm suprised Storm can't do INSERT
<lifeless> INSERT yes
<lifeless> INSERT INTO no
<lifeless> sorry, I should be clear INSERT INTO ... SELECT
<StevenK> lifeless: So it's an insert with a sub-select?
<lifeless> yes
<lifeless> it performs a query and inserts the result into a table
<StevenK> lifeless: Bug filed, #613300
<_mup_> Bug #613300: Storm needs to support INSERT INTO ... SELECT <Storm:New> <https://launchpad.net/bugs/613300>
<mwhudson> setting up the functional layer takes 25 seconds on this machine :(
<lifeless> hah
<lifeless> :(
<StevenK> lifeless: Anything else, re: that MP?
<lifeless> not at the level you were asking about
<StevenK> mwhudson: What is it, a coal-fired 486?
<lifeless> which was just 'why the raw sql'
<mwhudson> StevenK: netbook
<StevenK> lifeless: Well, the other question was is the ducking underneath and all that still necessary?
<lifeless> what do you mean 'ducking underneath' ?
<StevenK> lifeless: In regards to the comment at line 108 in the MP diff
<lifeless> ah
<lifeless> well while INSERT INTO...SELECT is used; yes.
<mwhudson> TWENTY FIVE SECONDS
<lifeless> spm: how about rt 40685 - is that blocked ?
<wgrant> mwhudson: What sort of hardware is that?
<cody-somerville> interesting
<mwhudson> wgrant: it's an aspire one, some atom thing at 1.66 ghz
<cody-somerville> https://code.edge.launchpad.net/django <-- the first branch lists two series, one of the series doesn't even belong to the django project - is that a feature or a bug?
<spm> lifeless: only on resourcing afaict
<lifeless> spm: ok.
<lifeless> spm: wanna bikkit!
<lifeless> spm: you might like to eyeball https://dev.launchpad.net/LEP/ReleaseFeaturesWhenTheyAreDone
<wgrant> mwhudson: Haha.
 * mwhudson having defeated zope again for the moment, retires for the evening
<wgrant> mwhudson: Did you get around to sending those branches off, or did the Atom defeat you?
<mwhudson> wgrant: i didn't send your branches, sorry
<wgrant> mwhudson: No worries.
<lifeless> wgrant: Are they all totally good to go /
 * StevenK kicks IArchiveSet.getByDistroPurpose()
<wgrant> lifeless: All except test-ddeb-matching are reviewed. I should probably get that one re-reviewed, since there were some changes made.
<wgrant> StevenK: Why?
<wgrant> StevenK: ArchiveCollection may interest you.
<j24> wgrant: thumper: I want a local bazarr management tool
<j24> and take a look on launchpad
<j24> it look good
<j24> but it's seems impossible to use for local projects management
<StevenK> wgrant: Because I'm calling it from a doctest, and it's printing a security proxied IArchive, and it gets called from the script, and it's None
<wgrant> StevenK: Awesome.
<StevenK> wgrant: FSVO
<lifeless> spm: any concerns about that wiki page from your perspective ?
<spm> lifeless: in stage 1, it might be worth raising the idea of switching app servers in/out of the pool of available services on the LB's
<lifeless> spm: isn't that whats done ?
<spm> nope
<lifeless> ...
<spm> tbh, short of config file changes and reloads of the LB's, it may not even be possible.
<lifeless> ok
<lifeless> I'm going to ostrich this one
<spm> heh
<lifeless> if it fuads
<lifeless> we'll stop moving through the process and address it
<spm> @ $job-1; we were able to switch servers on/off in the web ui; but as we only had 3 app servers to worry about; and rollouts were light; it wasnt hard to do so.
<spm> heh, and the idea of automated config changes to the running config of the LB's gives me the willies. switching set A in; out; B in etc. It may work; but *wince*.
<spm> it'd be nice if haproxy (it may, i just don't know) had some sort of remote push notify that a service was going down. vs it's current detect that service is down.
<adeuring> good morning
<wgrant> bigjools: Morning.
<mrevell> Hellooo
<bigjools> morning
<wgrant> bigjools: Any progress on the log parser setup?
<bigjools> I asked for it to be re-instated
<bigjools> about a week ago in fact
<bigjools> not sure if it got done though
<StevenK> wgrant: O hai?
<StevenK> wgrant: I'm working on moving initialiseFromParent() out of IDistroSeries, and I've stopped it creating archives that don't exist. What should it do if the debug archive doesn't exist?
<wgrant> StevenK: Hi.
<wgrant> StevenK: Why would iFP be creating archives?
<StevenK> It currently does
<wgrant> What.
<bigjools> it's bogus code
<StevenK> For example, ArchivePurpose.PARTNER
<wgrant> Ah, I guess it's designed to work cross-distro.
<wgrant> And it will need to soon... but I'd just drop that code until we know what it actually needs to be extended to do.
<StevenK> wgrant: I'm not sure I follow. Have iFP ignore debug archives?
<wgrant> StevenK: iFP is currently used within a single distro.
<wgrant> If there's no archive to publish to, there isn't one to copy from either.
<StevenK> Right
<wgrant> Hmm.
<wgrant> The current code is reasonable, though.
<StevenK> wgrant: Have you seen my MP?
<wgrant> StevenK: I looked at it around 6am this morning, so didn't really take in much...
 * wgrant looks again.
<wgrant> Hm.
<wgrant> Does it really belong in scripts/?
<wgrant> We have stuff like the copy infrastructure in there... but then we call it from outside. I think it should be somewhere else.
<wgrant> But anyway.
<bigjools> it's a script
<bigjools> it should live in scripts
<wgrant> StevenK: Ah, so you're fixing it to not blindly copy PARTNER any more?
<wgrant> In that case, yes, whitelist DEBUG along with PRIMARY.
<wgrant> They need to be copied together at all costs.
<bigjools> debug doesn't exist yet though does it?
<wgrant> It doesn't, no.
<wgrant> Oh, blah. That creates it even if there are no publications.
<wgrant> Oh, no. It doesn't.
<wgrant> So that's fine.
<bigjools> so the script should only copy ddebs if the debug archive exists
<wgrant> There are only ddebs to copy if the debug archive exists, so it won't copy them unless it exists.
<bigjools> exactly :)
<wgrant> bigjools: Ah, thanks for fixing that.
<bigjools> you're such a stalker :)
<wgrant> I just subscribe to (db-)devel!
<StevenK> Haha
<lifeless> wgrant: ping
<lifeless> wgrant: Can you find more memcaching bugs - I can't find a solid list of them.
<wgrant> lifeless: Grep the tree for tal:cache, take most of the results!
<wgrant> (I can't do it myself right now, sorry)
<lifeless> hah, lol, thanks.
<jml> hello
<lifeless> hai
<jelmer> moin
<bigjools> g'tag
<jtv> sinzui, hi!  Say, would you mind having a look at how the FakeLibrarian currently installs itself and say whether I'm on the right track?  Whether we want layers or test resources to take care of actually invoking it is for later.  WIP MP: https://code.launchpad.net/~jtv/launchpad/fake-librarian/+merge/30907
<lifeless> anyone from bugs around ?
<lifeless> jml: you wanted to catch up on testr; was there other stuff? should we have a brief call?
<jml> lifeless, just testr. need coffee before call.
<jml> lifeless, hello
<lifeless> hello
<wgrant> lifeless: I received an email about one of those branches, but not the other two.
<lifeless> no idea why :)
<wgrant> Yay.
<lifeless> jml: this is a core example of the sort of thing I was talking about re: storm etc - https://bugs.edge.launchpad.net/launchpad-foundations/+bug/251284
<_mup_> Bug #251284: Building a representation of one entry should not take more than 1 database request <api> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/251284>
<deryck> Morning, all.
<jelmer> hey Deryck!
<deryck> lifeless, you around?
<lifeless> hai
<deryck> so I'm a little aggravated that the caching of the hot bugs list as been held up as the poster boy for the horror of memcached.  But I certainly don't mind if the caching is removed.
<lifeless> sorry :( its only one case
<deryck> For the record, I did not do it to fix performance problems on the bugs home page.  You can see two MPs for bug 590992, where I fixed the root cause of that issue first and then added caching to try to buy overhead in times of trouble again (i.e. back when the db was on fire).
<_mup_> Bug #590992: https://bugs.launchpad.net/ubuntu times out consistently <qa-ok> <timeout> <Launchpad Bugs:Fix Released by deryck> <https://launchpad.net/bugs/590992>
<deryck> So I don't appreciate the "developers don't know how to cache" meme going around the bugs and email thread.
<lifeless> sorry!
<lifeless> It has definitely got overpersonal and a bit insulting
<lifeless> and thats sad, because I don't feel that way about our team
<deryck> I agree there are problems with memcached usage currently.  And perhaps it's better to focus our efforts on root cause of timeouts than getting the memcached infrastructure where it needs to be....
<deryck> and fixing specific bad uses of caching, of course, fine....
<deryck> but we'll never get memcached working if we don't try somehow.  and I think it is useful if used well.
<lifeless> I totally agree
<lifeless> I had tried to make that clear; its a tricky subject :(
<deryck> Yes, I agree it's tricky.  And I agree with your perceptions, but not your conclusions.  In other words, yes, we have issues, yes, we need to do better, and priorities need to be accessed.  But I don't think backing out the facility completely, thereby preventing any further use, is the right approach.
<lifeless> I've just re-read the thread, and I'm unhappy that you're unhappy - neither of my mails talks about motivation or knowledge at all.
<deryck> lifeless, "message and developer actions are mismatched" in response to memcached not being about timeouts implies developers motivations are just to fix timeouts....
<deryck> in the bug on this hot bugs issue you quote the great unsolved quote as if I don't know that and tell me to fix performance some other way
<lifeless> deryck: for that one I apologise wholeheartedly
<deryck> no worries, I'm not *that* upset about it. :-)  and look, I'm not trying to be a cry baby about it.  It's not that big a deal in the end....
<lifeless> deryck: it was out of line, I was a bit unhappy at it being low I think.
<deryck> fair enough.  I would rather us fix the memcached infrastructure to have some basic invalidation capabilities, rather than revert the usage.  I could use a feature flag to cache for edge now, when I couldn't before.
<lifeless> deryck: so, I have a few things I want to ensure; I want to ensure that root causes are fixed first. Its great that you're doing that; I know via direct observation that it is not universally the case.
<deryck> totally agree that we should fix that first.  I've not chatted with anyone to know others are doing differently, but I certainly believe you :-)
<deryck> And I'm happy to have this one case reverted if it makes everyone happier.
<lifeless> I want to ensure that we don't have coherency issues that users percieve - I'm all for decreased consistency to increase performance - but we can get a lot of mileage from just a little inconsistency
<lifeless> bah, perc*ei*ve - its midnight
<deryck> right.  Sorry to get you so late.  I appreciate you chatting with me about it.
<lifeless> I have a concern - possibly entirely unjustified - that we're missing a huge slab of infrastructure to do invalidation at all well : rabbit may help, but its pretty tricky knowing *all* the places that something can have been cached in.
<lifeless> anyhow, I'm not talking about pulling the plug from memcached in the datacentre; I'm talking about making sure that where we use it, its transparent to the user, and beneficial to us: we should be saving more time than we spend putting stuff in it during our current peak-load events
<deryck> ok, if we're talking about "let's use it better, here's how..." then I'm completely supportive of that.  I read "propose to turn memcached off Friday" as taking away the facility completely.
<lifeless> slightly hyperbole, and I didn't talk implementation details - I should have.
<lifeless> I think memcache is a wonderful tool (I'm upstream on squid FWIW - I *really* like my caches)
<bigjools> jml: I'd like a chat with you later about derived distros so I can bounce some issues off you if you're ok with that
<jml> bigjools, sure. a bit later though, not really feeling 100%
<lifeless> deryck: while we're talking
<bigjools> jml: no prob, me neither tbh.
<lifeless> deryck: https://bugs.edge.launchpad.net/launchpad-project/+bugs - search for 'caching' then click on next to get to the second page.
<deryck> ok, looking....
<lifeless> deryck: I see the first bug listed twice
<lifeless> and the 10th and 11th rows are duplicates too
<deryck> lifeless, because you're searching on a project group and the project is different.  See the project column.
<lifeless> yeah
<lifeless> is that deliberate?
<deryck> Honestly, I'm not sure.  It's been that way awhile, so I don't know.  There's a bug about duplicates in series or package listings, but not about project groups, I don't think.
<lifeless> if you don't consider it a feature, I can find-or-file a bug on it tomorrow
<deryck> lifeless, yeah, I think it would be better to list the bug individually, and have each project affected listed in the project column.
<lifeless> that would be really nice
<deryck> it makes no sense for us to list multiple projects, since we're mostly one project.  But for other project groups, it could be nice to see a bug affects multiple projects.
<lifeless> have you looked at the LEP/Search yet and made sure your use cases are well listed ?
<lifeless> I realise you're flat out :), but I do hope to move that forward soon.
<lifeless> deryck: I don't know if you've heard this, but apparently many users in the distro do their bug searching via +filebug
 * jml -> lunch
<deryck> lifeless, I did look at it.  I thought it was well covered but will look again.  I did know the distro used filebug.  I often do the same. :-)
<lifeless> deryck: how would you feel if I put up a patch consolidating the search logic for them at some point ?
 * deryck thinks....
<deryck> lifeless, I think that would be fine.  My only concern is that filebug can be a bit aggressive and show odd results sometimes that make you think "now how is that related?"
<lifeless> deryck: I think we've pretty much fixed that :)
<deryck> ah, right :-)
<deryck> lifeless, so then, yeah.  Please do it! :-)
<lifeless> well, ECycles, but I'll keep it in my scratch list
<lifeless> by which I mean, I won't commit to it, because that would impede someone else doing it, but if I happen to have some slack I'd certainly like to shrink the code base a little and that seems like a place with some overlap
<deryck> yeah, definitely.  I'm happy for it to be done, and fine if anyone does it.  I doubt another bugs team dev would have time for a while either.  Unless there own itch itches. :-)
<lifeless> who runs http://twitter.com/launchpadbugs ?
<stub> deryck: Should https://code.edge.launchpad.net/~stub/launchpad/bug-602936-hotbug-caching/+merge/31735 be rejected? I see no other way of addressing the bug in the short term, so it depends if you think the solution is better or worse than the problem (low priority, so I guess not much of a problem)
<lifeless> it seems very selective about what it retweets
<deryck> lifeless, hmmm, I didn't even know that existed.  Not sure who runs it.
<lifeless> :)
<lifeless> the miracles of doing a search to see if anyone complained about the filebug change
<lifeless> matsubara: hi
<lifeless> man, this is great, If only it wasn't 12:30
<matsubara> lifeless, hi there
<deryck> stub, no, I think reverting the memcached usage there is fine.  For some reason, I thought you were still working on memcached in the sense of trying to get a simple invalidation mechanism....
<deryck> if not, then yes, we should clearly remove this usage.
<lifeless> matsubara: have you seen the fresh https://dev.launchpad.net/LEP/ReleaseFeaturesWhenTheyAreDone
<matsubara> lifeless, nope
<lifeless> matsubara: if I can divert you for a few minutes
<lifeless> and ask you to look at it
 * matsubara reading now
<stub> I don't think we have gotten as far as deciding on an approach for invalidation, let alone ready to code.
<stub> I'm in favour of seeing what we can do with what we have, because that is less work ;)
<deryck> heh, fair enough :)
<matsubara> lifeless, ok. do you need any specific feedback on it?
<lifeless> matsubara: yes; where are you guys up to with respect to the shepard ?
<matsubara> lifeless, mars is leading that. Last I heard there's: https://dev.launchpad.net/QAShepherd and mars registered https://edge.launchpad.net/qa-shepherd
<lifeless> ah
<lifeless> I shall chat with gary and mars then
<deryck> bigjools, hi.  6 or 7 of the QA items for bugs are my markAsDuplicate work that was reverted, so I'm trying to get that back in today and qa'ed ASAP....
<lifeless> that seems rather more than I had expected.
<deryck> bigjools, as for the rest, I'll chase them up or nudge others today.
<mars> lifeless, have had a day or two of coding on it, and also hammered out a list of stories outlining the minimum viable feature set.
<bigjools> deryck: thanks man.
<deryck> np!
<lifeless> deryck: for that thing, you need a new storm; danilo says our custom one is just a few commits from trunk
<bigjools> and the thought of QA makes me hungry
<lifeless> deryck: so grabbing the latest release if it has the caching bug fixed will do it
 * bigjools -> lunch
<lifeless> deryck: I'd say ask jkakar but hes on a plane, therve may know.
<lifeless> mars: well, its nearly 1am, but perhaps you gary matsubara and I can have a call right after the team leads meeting ?
<deryck> lifeless, ah, ok.  I will.  I was also experimenting with other ways to force the update.  transaction.commit() works, muhahahah! ;)
 * deryck kids obviously
<lifeless> :P
<lifeless> deryck: it was a pita - spent the whole day tracking the thing down :(
<lifeless> took a bit of demonstrating to convince the storm guys :)
<mars> lifeless, that is at 1400?
<deryck> lifeless, yeah, sorry.  I thought it was well tested and working well when I left it.   When you say "the caching bug," is there a bug number for storm?
<lifeless> deryck: oh, I dunno if we got that far
<lifeless> deryck: uhm, ask therve, its my best advice
<deryck> lifeless, ok, thanks.
<lifeless> mars: I don't know my 7am. Its nearly my 1am now.
<mars> lifeless, oh, ok.  So we should talk in 6-8 hours then.  Perhaps your 9am, depending on matsubara's EOD
<mars> TL call is 2pm here
<matsubara> mars, lifeless: after the TLs meeting sounds good to me
<lifeless> thanks, that will be great.
<lifeless> mars: < 9am would be good, asiapac reviewers meeting is at 930
<lifeless> anyhow, ciao, sleep
<mars> Ursinha, good morning!  I had a question about the qa-tagger/shepherd handoff: the shared DB would contain revno/branch URL, right?
<Ursinha> mars, yes, and mentioned/linked bugs, if possible
<Ursinha> well, that's possible :)
<mars> Ursinha, so branch name and a list of linked bugs?  Or the revno and branch name?  (I can get the list of linked bugs from the branch, can't I?)
<Ursinha> mars, revision number, linked bugs and branch name
<Ursinha> mars, yes, you can, but I think we could avoid requiring checking the branch again if the tagger can do that already
<mars> ok, thanks
<Ursinha> s/can do/will do/
<gary_poster> benji___ (________!) mars matsubara Ursinha (stub): kanban now, mumble in 2
<wgrant> mars: When you have a moment, could you look at my changes to https://code.edge.launchpad.net/~wgrant/launchpad/test-ddeb-matching/+merge/31482 and land it?
<mars> wgrant, certainly
<maxb> Do the Canonical ISD hackers have any IRC presence? I was hoping to poke them about potentially making their workalike to launchpad-dependencies public
<wgrant> I've looked for one several times, but have always failed.
<wgrant> AFAICT they do not have a public presence of any kind.
<mars> maxb, maybe ask flacoste about if they might open it up?  He knows all :)
<flacoste> mars, maxb: i'll tell stuartm
<maxb> flacoste: thanks - I believe I have located the project: https://code.launchpad.net/canonical-isd-web-framework - and by subtle hints in the UI I can tell there's a private branch there :-)
<Ursinha> lifeless, hello
<james_w> How would I test that a row was removed from the DB?
<james_w> remember it's id and then try and get it and catch NotFound?
<jelmer> james_w: that seems like a reasonable approach
<Ursinha> bigjools, hi :)
<Ursinha> bigjools, have you seen this before/do you know what this means? https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1672EA4370
<bigjools> ummm
<bigjools> that's an odd one
<bigjools> Ursinha: I suspect he had the page open for a while and it finished building in the mean time
<bigjools> because if you go to the page now it redirects back to the build
<Ursinha> bigjools, I've seen some of those last days, but couldn't reproduce
<bigjools> Ursinha: yes, that's exactly what I see
<Ursinha> bigjools, want me to file a bug?
<mars> wgrant, I haven't forgotten your branch, but I have been bound by meetings
<bigjools> Ursinha: yeah, we can make a nicer message to the user
<Ursinha> bigjools, bug 613480, sorry for the poor bug title
<_mup_> Bug #613480: CannotBeRescored OOPS instead of a more friendly message <oops> <Soyuz:New> <https://launchpad.net/bugs/613480>
<bigjools> that's cool, thanks
<Ursinha> bigjools, np :)
<bigjools> sinzui: is "version" mandatory for a distroseries?
<sinzui> I think so
<sinzui> bigjools, remember that version is is in the URL, it is what we traverse to see the istroseries
<bigjools> sinzui: the name is in the url not the version
<bigjools> sinzui: I am wondering if it could be optional for derived one-shot distroseries
<sinzui> bigjools, I think we need to answer this in the context of bug 547082 and bug 419788
<_mup_> Bug #547082: LTS series are not designated as such <Launchpad Registry:Triaged> <https://launchpad.net/bugs/547082>
<_mup_> Bug #419788: Display name for distribution series doesn't follow vendor conventions <Launchpad Registry:Triaged> <https://launchpad.net/bugs/419788>
<sinzui> bigjools, some of the discussion to address these bugs imply version is guaranteed. If we are certain version is optional in these cases, then we can make the field optional
<bigjools> sinzui: that latter bug only applies to Ubuntu
<sinzui> not so
<bigjools> the way it's written makes it sound like it's for all of them
<sinzui> the issue apparently related to debian
<sinzui> I am wary of making series parent None per another bug. I think we should admit the Lp is for Ubuntu and sometime debian.
<bigjools> I can't believe that all distros we'd ever host would somehow want to change from a codename to a version when released
<bigjools> well, the changes I am making is going to make it for Ubuntu and a metric asshat load of derivatives :)
<sinzui> Lets be honest. We host one distro. There are several derivative distros like baltix that do little because we do not let them do anything
<bigjools> we should not allow ourselves to be painted into a corner by Ubuntu
<bigjools> at some point in the very near future we will be hosting more derivatives
<sinzui> We are still getting complaints from fedora and gentoo user that they cannot register a sourcepackagename. We should remove these distros we registering in our exuberant maddness thinking we could support them in a few months of creating Ubuntu
<sinzui> bigjools, We have IDerivativeDistribution now because I refused to let Ubuntu dictate how other distros work
<bigjools>  /o\
<bigjools> sinzui: can you please take a look at https://dev.launchpad.net/LEP/DerivativeDistributions
<bigjools> and if you have time, https://dev.launchpad.net/LEP/DerivativeDistributions/UserTestingRound1
<bigjools> but I am currently making mockup changes based on user feedback
<bigjools> I feel a great disturbance in the Force around making this fit for Ubuntu-derived distros, and Ubuntu's derivation from DEbian
<jelmer> gary_poster: Hi!
<jelmer> gary_poster: As promised yesterday I've just sent an email to the list about the PPA, EC2 images.
<barry> bigjools: hi
<bigjools> barry: hi
<barry> bigjools: i'm happy to talk about anything you want, including the net-snmp builder hang.  i'd like to get some lunch first though.  will you be around for a little while?
<bigjools> barry: I leave in 1.5 hours
<barry> bigjools: ok.  i will ignore the rumble, if this is a good time for you
<bigjools> barry: one thing though, for your FTBFS display, you should use the "show builds" page
<bigjools> your "want to know newer versions" is an interesting one
<barry> bigjools: is that the "view all builds" link, i.e. +builds?
<bigjools> and I can see why you were interested in my work on derivatives :)
<bigjools> yes, +builds
<barry> :)
<bigjools> it will fit your use case perfectly
<bigjools> barry:  see https://dev.launchpad.net/LEP/DerivativeDistributions
<deryck> adeuring, r=me, with minor changes.
<adeuring> deryck: thanks!
<deryck> np
<bigjools> barry: BTW are you the one making lots of calls to getBuildRecords on the API at the moment?
<bigjools> I see thousands of soft oopses :(
<barry> bigjools: it's close, but not perfect.  i.e. i really want a !successfully-built view, but currently there are a number of failing states you'd have to select individually.  with the script i can see all the failures and (optionally) the reasons
<bigjools> barry: ah ok interesting
<bigjools> we should fix that
<bigjools> can you file a bug please!
<barry> bigjools: i read that LEP, and indeed i'm very interested in that :)
<barry> bigjools: will do!
<barry> bigjools: re: gBR.  could be.  i only run those scripts once in a while though (< 4 times a day at most)
<bigjools> ok
<barry> bigjools: bug 613501
<_mup_> Bug #613501: Wishlist: one !successful-built status for a ppa's +build page <Launchpad itself:New> <https://launchpad.net/bugs/613501>
<barry> bigjools: do you want to look at the net-snmp builder hang?
<bigjools> barry: I suspect it's a bad job, it stuck on 2 arches
<bigjools> we reset the amd64 one, let's see if it repeats
<barry> bigjools: cool, thanks.  is there anything i should do about stuff like that in the future, other than ping you or a losa?
<bigjools> barry: so ideally not me in the future, but I appreciate there's not many people with the knowledge :)
<barry> bigjools: :)
<bigjools> if a losa would like to look at this problem with me now we can do some knowledge transfer
<barry> bigjools: cool.  just let me know what the s.o.p. is and i'm happy to follow it
<bigjools> I don't think this has a sop :)
<barry> awesome.  i love to blaze trails, in both senses of the word :)
<bigjools> barry: you mean leaving a smoking mess behind you? :)
<Chex> bigjools: I can work with you on this
<barry> bigjools: i seem to be (too) good at that
<bigjools> barry: :)
<bigjools> Chex: great!  So, we have a problem where a build was on a builder for 2 days.  In this case that's not expected so we need to work out what's going on.
<bigjools> barry: how long would you normally expect it to take?
<barry> bigjools: i'm not sure.  i haven't built that package locally (it was a resync request).  let me do that and see how long the sbuild takes
<bigjools> Chex: therer are 2 builds that look stuck, we reset the amd64 one to see if it does it again.  The i386 one is here: https://edge.launchpad.net/~pythoneers/+archive/py27stack4/+build/1901035
<Chex> bigjools: ok, looking there
<bigjools> Chex: there's usually one of three outcomes to this problem: 1. the job really does take a long time, 2. the job is stuck, 3. the builder has got stuck
<Chex> bigjools: I have to break soon for a shortish meeting, but can pick up with you uafter that
<bigjools> Chex: ok ping me when you're back.  I will be gone in an hour though.
<Chex> bigjools: ok, so how do I know its one of the 3?
<bigjools> Chex: for (1) we can ask a packager how long he expects it to take normally
<bigjools> Chex: for the others, the first thing is to reset the builder which forces the build onto a different builder.  If it does it again, the job is a bad one and we need to kill it.  That needs SQL at the moment until we do some UI work.
<bigjools> it's nearly always a bad job.
<bigjools> Chex: do you know how to reset the builders?
<barry> Chex, bigjools i'm building net-snmp locally now
<Chex> bigjools: no I dont find any wiki notes on how to do that
<barry> Chex, bigjools oh shit, i know what the problem is
<barry> Chex, bigjools you might as well kill the build, it will never finish
<Chex> bigjools: I need to step away for my meeting now, but I will pick up with you again in a bit, hopefully before you leave
<bigjools> Chex: ok, if it's a non-virtual builder we have to get a GSA to reset it.  if it's virtual (a PPA builder) we can run the same reset script that the buildd-manager runs.
<bigjools> Chex: ok
<barry> Chex, bigjools i saw this on monday, but doko has not yet fixed it.  'apt-get install python-pkg-resource' from ppa:doko/toolchain will hang forever.  i think it's waiting for stdin.  this is clearly my biggest priority to fix
<bigjools> barry: ewwwwww :/
<barry> bigjools: indeed. :( :(
<barry> it's basically a blocker now for everything else i need to do
<barry> so... i'm going to get some lunch, and then work on this as a top priority.  it's possible other py27stack4 and py27stack5 builds will hang for the same reason.  feel free to kill them all, and i will not request retrys/resyncs until i've fixed this particular problem.  i may request a priority bump on this package at that point, or get doko to do so
 * barry -> lunch
<bigjools> barry: I'm going to disable your PPA until we kill the builds
<bigjools> the farm is DOSed with stuck builds
<barry> bigjools: do the losas know how to re-enable it?
<bigjools> yes
<bigjools> but we'll also need to kill all the builds
<barry> bigjools: cool.  yes, kill them all.  i've emailed doko (he's not online atm).  i'll ping the losas when i know i have a fix for this blocker
<Ursinha> bigjools, hey, I have another oops question for you
<Ursinha> bigjools, I've seen timeouts in DistroSeries:EntryResource:getBuildRecords, I wonder if you're aware of those
<Ursinha> bigjools, https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1676EB950 and
<Ursinha> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1676EB2797
<jml> jelmer, hello
<jelmer> jml: Hi
<jml> jelmer, you might want to try out the testrepository branch https://code.edge.launchpad.net/~jml/testrepository/show-failures-incrementally-613152/+merge/31765 to see if it addresses your bug 553240
<_mup_> Bug #553240: Please provide --passthrough / --tee option to 'testr run' <testr-run> <Testrepository:New> <https://launchpad.net/bugs/553240>
<jelmer> jml: ah, nice!
<jelmer> jml: I'll give it a whirl.
<jml> jelmer, thanks.
<bigjools> Ursinha: yes, the foundations team are fixing it, there's a performance issue in lazr.restful
<Ursinha> bigjools, ah, nice
<Ursinha> bigjools, last oops for you:  https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1676K716
<mars> could someone please lp-land lp:~wgrant/launchpad/test-ddeb-matching ?  My lp-land command is busted :(
<jelmer> mars, sure
<mars> thanks jelmer
<jelmer> jml: I guess it doesn't display anything for successful testruns?
<jelmer> for individual tests I mean?
<jml> jelmer, right. that's the current behaviour of "testr load"
<jml> jelmer, the difference in this branch is it displays errors/failures as it gets them.
<bigjools> barry: before I enable your PPA again we need to kill all those pending jobs, sorry.  I can't take the chance of it DOSing the farm again.
<jelmer> jml: Yeah, I agree that's a big improvement over what we had previously.
<jelmer> jml: Like that one time I ran "testr run" on lp only to find out I had broken the testfactory *after* about an hour of running tests...
<jml> jelmer, :(
<jelmer> mars: It seems busted here as well
<jelmer> mars: ValueError: Cannot determine Bazaar host. "https://api.edge.launchpad.net/1.0/" not a recognized Launchpad API root. is what I get
<mars> Heh, I get AttributeError: 'Entry' object has no attribute 'source_branch'
<jelmer> mars: You're specifying the branch URL rather than the MP URL perhaps?
<mars> jelmer, yep, that might do it
<mars> argh, and now ec2 test -b lp:~wgrant/launchpad/test-ddeb-matching breaks too
<mars> why don't all of our commands take the lp:~foo form by default?
<jelmer> mars: why the -b ?
<mars> jelmer, because it is not my branch.  I want to test William's branch.
<jelmer> mars: IIRC I've run "ec2 test lp:~wgrant/..." without problems before.
<jelmer> -b seems to be for including extra branches in sourcecode/
<mars> argh, yep
<james_w> anyone seen http://paste.ubuntu.com/473160/ during test setup?
<mars> jelmer, I would have preferred '-s', for '--sourcecode-branch='
<mars> james_w, might be related to some work with the twisted logger henning did recently?
<jelmer> mars: I agree that'd make more sense, given the -b option to "ec2 land" also takes a branch but has a different purpose.
<james_w> mars: I think it's environment as it just started happening in two different branches at the same time, without me merging in anything.
<james_w> it's just that the output is so damn unhelpful
<mars> james_w, yes.  Check /var/tmp for librarian pidfiles and the like
<henninge> mars, james_w: that is not related to my work because that just changed buildd-manager.tac.
<james_w> I can run daemons/librarian.tac under ./bin/twistd without an issue
<mars> james_w, and maybe kill the librarian log files too, and check your process tree for dead librarians
<mars> henninge, ok, thanks for the clarification
<james_w> mars: stale librarian process apparently, thanks.
<james_w> any idea where I would look to get it to say "port already in use" rather than the junk you get now?
<mars> start with the librarian.tac file, see if you can work down through the callstack
<jml> james_w, I'll race you :)
<mars> something must be catching and masking the real error
<jml> oh man, TacHandler
<jml> what's happening is it's spawning a subprocess and then looking for a magic token in the logfile
<jml> without regard to what the process dumps to stderr/out
<jml> tachandler should be completely destroyed and rewritten as a series of good patches to Twisted.
<james_w> jml: tachandler runs a subprocess?
<jml> yeah.
<jml> it spawns "twistd" directly
<james_w> TacTestSetup?
<james_w> that does read something from stdout/stderr though
<jml> james_w, yeah, it does
<jml> but only oncee.
<james_w> jml: something like http://paste.ubuntu.com/473176/?
<jml> james_w, that ought to do it
<bigjools> barry: ping
<bigjools> barry: going to nobble all outstanding builds in your 2 PPAs
<bigjools> last chance to stop me
<barry> bigjools: do it
 * bigjools hits the red button
<bigjools> barry: it's done
<bigjools> you can re-enable your PPAs yourself
<bigjools> in the +edit page
<bigjools> g'night!
<barry> bigjools: nod, and g'nite!
<bigjools> and don't break the build farm again or I'll break your legs!
<bigjools> ;)
<james_w> jml: didn't fix it
<jml> james_w, a shame.
<james_w> twistd returns 0 and doesn't print anything?
<jml> oh, huh.
<jml> it might.
<jml> easy enough to check, I guess.
<jml> actually, it definitely would
<jml> it daemonizes before opening the port.
<james_w> ah, it daemonizes
<jml> yeah, that's what the -y is for, also why tachandler does crazy logging stuff
<james_w> so we need to wrap the .tac in a big try/except and try and log the exception before raising it?
<james_w> twistd doesn't have any support for backchannel fds after it has forked or anything?
<jml> james_w, no, I don't think it has such support.
<james_w> using twisted.python.log and calling log.error in an except block around the entire .tac doesn't work
<james_w> so I'm at a loss
<jml> james_w, I'm sorry :(
<james_w> it only means that we will have to continue to scratch our heads when daemons fail to start
<jml> right
<jml> the _real_ solution to all of this is to give Twisted more APIs to do what twistd does.
<jml> james_w, on another note, do you need a hand landing https://code.edge.launchpad.net/~james-w/launchpad/no-more-sampledata-1/+merge/31493
<Ursinha> sinzui, hello
<james_w> jml: nope, just about to do that now the prerequisite is landed
<jml> james_w, cool :)
<Ursinha> sinzui, do you have a clue on why are we having lots of timeouts in MailingListApplication:MailingListAPIView recently? on staging, that is
<Ursinha> sinzui, I also see a bunch of informational oopses, DisconnectionErrors, related to MailingListApplication:MailingListAPIView, but no idea if they're related
<sinzui> Ursinha, recently? They go back months
<sinzui> There are timeouts syncing large numbers of users
<Ursinha> sinzui, I meant much more than usual
<sinzui> Ursinha, Reduced the number in a branch he landed a few weeks ago
<Ursinha> right
<Ursinha> sinzui, I saw lots of timeouts like this one, two days ago: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1675S1478
<sinzui> yes, I see those everyday
<Ursinha> my question is, is this raise in the number "expected" somehow
<sinzui> I am ignoring them
<sinzui> They are not a high priority at this time
<sinzui> I did ask an engineer to fix the issue, and he did take 3 days to reduce the issue
<sinzui> Ursinha, We see a rise because we decided that syncing staging to lpnet is a priority. We used to sync once a month, (every 6 weeks in reality) We now sync every week
<Ursinha> sinzui, ah, that explains
<sinzui> Ursinha, it is a scary explanation though...it means staging has never been able to deal with the sync
<Ursinha> sinzui, I see that, I see those timeouts since I joined canonical, I guess
<sinzui> Ursinha, This is the second implementation of the sync. This one is designed to recover from timeout failures because we believe timeouts were happening in lpnet and that would block all other mailman sync procs
<sinzui> Ursinha, There was a proposed spec (salgado maybe) about getting efficient user sets. I think it was intended to address situations like this
<Ursinha> sinzui, hm, I see
<lifeless> moin
<Ursinha> hey lifeless
<lifeless> hey Ursinha
<lifeless> 'pong'
<Ursinha> :)
<lifeless> sinzui: https://bugs.edge.launchpad.net/launchpad-foundations/+bug/74680 is the only accountmerge bug I can see atm
<_mup_> Bug #74680: +accountmerge page should present the option to discard the merge request <Launchpad Foundations:Confirmed> <https://launchpad.net/bugs/74680>
<sinzui> Oh sweet, a new bug for me to move into launchpad-registry
<lifeless> :)
<jml> lifeless, please review my testr branches :)
 * jml runs away
<lifeless> jml: thanks
<lifeless> jml: also for the reminder to look at incremental and unblock you
<jml> lifeless, I've actually already come up with a patch
<sinzui> lifeless, I am reporting a new timeout bug for person merge. I think the one I care about was marked as a dup and some aspect of the master bug lead someone to mark it as closed.
<lifeless> oh cool
<lifeless> sinzui: makes sense to me.
<lifeless> so, gary_poster, mars : I want to catch up on the qa shepard stuff; there's now a calendar slot for a voice call in a bit; or would now be better?
<lifeless> I had thought to have matsubara or Ursinha there too, given its for qa :)
<gary_poster> lifeless, mars: now is fine with me
<mars> lifeless, either for me
<mars> lifeless, mumble?  Foundations channel?
<lifeless> we'll give mumble a shot, though NZ + mumble generally equals epic fail
<gary_poster> heh
<gary_poster> skype is fine too
<Ursinha> I'm there already
<mars> lifeless, I thought you were in Perth?
<lifeless> can you hear me ?
<lifeless> I can hear you ok :P
<Ursinha> lifeless, gary_poster, huge echo
<lifeless> I've turned my mic off for now
<Ursinha> thanks :)
<lifeless> project exists
<lifeless> mars: ^
<matsubara> for some weird reason I can't connect on mumble
<deryck> sinzui, ping
<lifeless> gary_poster: mars: https://dev.launchpad.net/LEP/ReleaseFeaturesWhenTheyAreDone <-
<lifeless> stage 2 there is what we're looking at
<sinzui> hu deryck
<mars> lifeless, Ursinha, gary_poster, https://dev.launchpad.net/QAShepherd
<lifeless> gary_poster: please do
<lifeless> (avoiding my mic for your sanity)
<lifeless> gary_poster: small lemma; its a revision we have, not a branch right? because buildbot lands stuff in stable asynchronously from qa happening ?
<lifeless> mars: right
<lifeless> mars: thats exactly what I was thinking
<lifeless> mars: for now, I only want to *aim* at being able to tell the losas what to deploy
<Ursinha> lifeless, mars, gary_poster, matsubara's internet just dropped
<Ursinha> and he's back, it seems :)
<matsubara_> and it's back again. minor hiccup it seems
<gary_poster> mars, Ursinha, matsubara_ mumble is a Zope question fwiw
<gary_poster> I don't think you need to hang around unless you want to
<Ursinha> ah, ok gary_poster
<lifeless> yeah, new topic :)
<lifeless> sorry guys
<mars> lifeless, first two items in place.  The old list didn't have end-user concerns first: oversight on my part.  https://dev.launchpad.net/QAShepherd
<lifeless> I think thats great
<lifeless> I'd be happy with just a number coming back from a DB :P, but having it as html is ok too - we can screen scrape to get the revision
<lifeless> [mostly teasing there, the use cases are fine]
<mars> lifeless, regarding the project: yes, it's set up.  I'm waiting on getting a shared branch so we can use the LP merge workflow.  The other alternative is to run Tarmac locally.
<lifeless> mars: what do you mean ?
<lifeless> [what is a shared branch in this context]
<mars> lifeless, I have ownership of mainline right now.  It would be nice if launchpad-pqm owned it instead
<mars> and if Tarmac could handle landing
<mars> Since there will be three people hacking on it
<mars> simultaneously
<lifeless> ok, so you're blocked on two things
<mars> but I can see about running Tarmac here instead
<lifeless> a) a branch for 'trunk'
<lifeless> and b) a tarmac in the D/C
<lifeless> if I may suggest somethinhg
<lifeless> create a team - lp-qa-shepard-committers
<mars> we already have that: lpqateam
<lifeless> no
<lifeless> In that team, put everyone that should be able to *bzr push* to trunk.
<mars> right
<lifeless> and make the trunk branch yourself, owned *by the team*, not by *launchpad-pqm*
<mars> yep
<lifeless> that gives you a)
<lifeless> and the branch url will be stable when you get a tarmac instance in the D/C
<lifeless> because launchpad-pqm will be part of the team, you simple take yourself, Ursinha & matsubara_ out of the team at that future date
<lifeless> its a bit of a bug our current lp branches are owned by launchpad-pqm, rather than by a team
<lifeless> oversight some years back
<mars> It might easier for now if I just set up Tarmac locally.  That way it is automated, but there are also no accidental overwrites from multiple pushers.
<lifeless> it should be clear why its not the lpqateam
<lifeless> mars: bzr won't permit overwrites
<lifeless> mars: you can do what I suggest *and* setup tarmac locally.
<mars> lifeless, so not reusing lpqateam - is that so commit access can be controlled later via team membership?
<lifeless> yes
<mars> so a hack then
<lifeless> using the model
<lifeless> is lpqateam a private team ?
<mars> https://edge.launchpad.net/~launchpad-qa
<mars> just restricted
<lifeless> right, its moral owner, not committers
<mars> committers actually
<lifeless> I don't think its a hack, given LP's branch permission model, to have a -committers team : many many projects do this.
<lifeless> mars: there is a bot in there
<mars> lifeless, yep, but you also need to be a member to work on a few of the QA projects
<mars> I have no idea why the bot is in there
<lifeless> mars: anyhow, its a suggestion. Its what I'd do - if its not helpful, thats ok, don't do it :)
<mars> ok, well if overwrites from multiple team members are not a problem, the great - that's what I would have liked to do in the first place
<mars> thanks lifeless
<lifeless> mars: set append_revisions_only=True in the branch.conf
<lifeless> and it will also prevent mainline wobbles
<lifeless> (bzr help configuration)
<lifeless> Ursinha: lp-oops being openid is nice, but its a _very_ short timeout - like only an hour or so : can we make it something reasonable, like LP? (several months would be nice :P)
<matsubara_> lifeless, losas control that
<lifeless> matsubara_: ah
<Ursinha> hehe
<lifeless> sinzui: I've linked the bug in the oops for you
<mars> matsubara_ or Ursinha, are you able to push code to the ~launchpad-qa team space?
<Ursinha> mars, yes
<matsubara_> mars, I don't think I tried, but I assume I can
<Ursinha> if you're subscribed to the branch, I guess you can
<Ursinha> don't remember trying to push to a branch without being directly subscribed to it
<mars> ok, I'm trying to push a branch, lp:~launchpad-qa/qa-shepherd/devel, and it spits it back at me saying "you don't have permission to do that"
<mars> even though I am a member of the team
<lifeless> mars: its because of the branch policy
<lifeless> mars: you need the right permissions on qa-shepard
<lifeless> losa ping
<Ursinha> lifeless, which would be the right permissions in that case?
<lifeless> Ursinha: you need a private branch policcy that permits anyone in the launchpad-qa team to make a new branch; but we're about to open it so that won't matter at all in a few minutes
<lifeless> Ursinha: as soon as my losa ping is answered ;P
<Ursinha> lifeless, hehe, right
<mbarnett> lifeless: hellos
<lifeless> hellos
<lifeless> we have a project, qa-shepard
<lifeless> that is currently canonical-private, we'd like to open it please ;)
<mbarnett> lifeless: when you say open, you mean set the default branch visibility to public?
<lifeless> yeah, remove all the privacy-bits
<lifeless> open bugs, open branches
<mbarnett> lifeless: i am not seeing that project
<lifeless> mbarnett: https://edge.launchpad.net/qa-shepherd
<mbarnett> ah
<mbarnett> sheepherd
<mbarnett> lifeless: Default branch visibility for all branches in qa-shepherd  is Public.
<lifeless> mbarnett: great, thanks.
<lifeless> mars: should be good to push now.
<mars> lifeless, cool, thank you
<Ursinha> that gives us the ability of using lp:qa-shepherd again :)
<lifeless> hey
<lifeless> is there a wiki page that documents db patch workflow ?
<mars> no idea
<thumper> morning
<mwhudson> i'm not the only person who gets mails from ec2 with attachments doubly gzipped am i?
<mwhudson> thumper: good morning
<wgrant> mwhudson: Morning.
<mwhudson> hello
<mwhudson> so wifi doesn't entirely work on this laptop, apparently
<wgrant> mwhudson: You reviewed https://code.edge.launchpad.net/~wgrant/launchpad/per-archive-build-debug-symbols/+merge/29671 yesterday, but didn't click the button, apparently.
<wgrant> Is it using iwlagn? I find that flakes out sometimes for me.
<wgrant> But normally only once it's been up for a week or so.
<mwhudson> how do i tell?
<mwhudson> this had been up for a few days with a few suspends
<wgrant> 'lsmod | grep iwl' should give you the driver, assuming it's Intel, which it probably is.
<mwhudson> wgrant: reviewed
<wgrant> mwhudson: Thanks. Can you ec2 it yet?
 * ajmitch had that happen last night with iwlagn, had to rmmod & modprobe again
<mwhudson> wgrant: it's ath9k
<mwhudson> so that explains the flakiness probably
<mwhudson> wgrant: yes
<mwhudson> well maybe
<mwhudson> boto.exception.BotoServerError: BotoServerError: 500 Internal Server Error
<wgrant> Yay.
<mwhudson> seems a bit better now
<james_w> hi mwhudson
<mwhudson> james_w: hi
<james_w> did you see that the vostok branch failed?
<mwhudson> wgrant: ec2 is headless
<mwhudson> james_w: yeah
<mwhudson> i guess i should do something about that
<wgrant> mwhudson: Thanks.
<james_w> the failures didn't mean a lot to me
<james_w> well, the second did, but the first didn't
<mwhudson> webapp-publication was pretty clear
<mwhudson> some of the others were pretty mysterious
<mwhudson> i haven't re run them locally yet
<wgrant> mwhudson: 25 seconds? :P
<mwhudson> yeah\
<wgrant> Can someone please ec2 https://code.edge.launchpad.net/~wgrant/launchpad/bug-612157-ppa-quota-ddebs/+merge/31471, https://code.edge.launchpad.net/~wgrant/launchpad/test-ddeb-matching/+merge/31482, and https://code.edge.launchpad.net/~wgrant/launchpad/bug-241129-queue-binary-scaling/+merge/31604?
<lifeless> wgrant: Didn't I do those ?
<wgrant> lifeless: Two of them, but they vanished.
<wgrant> Well, you did three, two of which are in that set because they vanished.
<james_w> wgrant: I will
<wgrant> james_w: Thanks.
<james_w> wgrant: ppa-quota-ddebs might want a merge with devel
<james_w> my apologies for that
<lifeless> thumper: https://code.edge.launchpad.net/~jml/testtools/patch-310770/+merge/31666
<lifeless> thumper: that used to have a diff; I think perhaps we're updating diffs after things merge now? So you can't see what was merged :(
<wgrant> james_w: Heh, yes, true.
<wgrant> james_w: Is the other one going to land soonish?
<james_w> wgrant: my next sampledata branch?
<wgrant> james_w: That's the one.
<james_w> wgrant: the next is in ec2 now
<james_w> I'm currently churning through fallout from making STP return proxied objects
<wgrant> Ah.
<lifeless> aaron is about to land a 'make it optional' patch
<james_w> one of the things that I'm sad about is requiring more tests to be in LPZopelessLayer, where DBFunctional was fine before.
<wgrant> I think james_w is about to squash most of the remaining warnings, though.
<james_w> wgrant: does each SPR require a PackageUpload?
<wgrant> james_w: Only for some purposes.
<james_w> right
<wgrant> Lots in production don't.
<wgrant> So we know it works.
<wgrant> (gina doesn't create them)
<james_w> ok, Julian said it should
<james_w> I'll probably revert that change to the factory, but leave it in STP, and then maybe go through another time and push it up in to the tests
<wgrant> It doesn't.
<wgrant> It'd need to forge a changesfile, for one thing.
<james_w> sorry, I meant that Julian said that all test code should ensure that each SPR had a PU
<james_w> though I may have misread
<wgrant> james_w: Conflict resolution pushed, and the other two are OK.
<wgrant> james_w: Most tests don't need a PU, so I don't see the point.
<james_w> ok, I'll back that change out of the factory, but leave the behaviour of STP the same
<james_w> this branch is already big enough as it is
<wgrant> Yep.
<lifeless> wgrant: so packagepublication
<wgrant> lifeless: You mean publishedpackage?
<lifeless> wgrant: can you suggest, in bug ..
<lifeless> wgrant: yes
<lifeless> https://bugs.edge.launchpad.net/malone/+bug/602360
<lifeless> and
<_mup_> Bug #602360: timeout on source package bug filing page <timeout> <Launchpad Bugs:Triaged> <Ubuntu:Invalid> <https://launchpad.net/bugs/602360>
<lifeless> https://bugs.edge.launchpad.net/malone/+bug/609012
<_mup_> Bug #609012: +distrotask timeout <timeout> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/609012>
<lifeless> https://bugs.edge.launchpad.net/launchpad-answers/+bug/608037
<_mup_> Bug #608037: timeouts in Question:+edit <oops> <timeout> <Launchpad Answers:Triaged> <https://launchpad.net/bugs/608037>
<lifeless> all on the same unmaterialised view
<lifeless> stub says 'nuke the use, or we have to materialise it.'
<wgrant> So, anyone using that is probably buggy.
<mwhudson> wgrant: my recipe build didn't build yesterday
<mwhudson> it says 24 minutes to build now though!!
<wgrant> lifeless: I need to leave in a sec, but I'll look on the train.
<wgrant> mwhudson: Isn't the build farm great?
<mwhudson> wgrant: it's certainly popular
<lifeless> wgrant: thanks!
<james_w> I have yet to have a recipe build
<james_w> sorry, an automated recipe build
<mwhudson> yeah certainly none of them
 * wgrant â uni
<sinzui> lifeless, I think the answers bug can be addresses by snapshots. I was going to work on it this or next weekend.
<lifeless> sinzui: snapshots ?
<sinzui> on a creation or change event objects  are snapshotted (copied) so that we can report changes to the object in logs and emails....
<sinzui> lifeless, We often see non-intrinsic data, often lists in the snapshot that we know cannot have been changed.
<lifeless> sinzui: oh right, so to paraphrase 'look at less data when snapshotting'
<lifeless> not 'add a snapshotting facility'
<lifeless> ?
<sinzui> right. as a rule anything we export as a collection field should not be snapshotted
<lifeless> sinzui: that would be cool
<lifeless> sinzui: I was simply correlating the same view showing up in lots of places
<sinzui> ah
<lifeless> lots of timeouts, I mean
<sinzui> doNotSnapshot() is needed on lots of fields that cannot be directly modified on a object. We also use it when we see a shortlist exception
<lifeless> is that a decorator ?
<sinzui> no, it is a function we wrap around field
 * sinzui looks for example
<lifeless> I'll grep
<lifeless> and follow my nose - thanks for telling me about it
<sinzui> I used it recently to stop api timeouts working with teams
<lifeless> should collections auto-do-this ?
<thumper> lifeless: it shouldn't be updating the diff after it is merged
<lifeless> thumper: I agree
<thumper> lifeless: as in the code says it shouldn't
<thumper> lifeless: so something strange happened
<thumper> lifeless: probably a race condition
<thumper> lifeless: branch pushed, update diff job created, merged, pushed, diff job runs
#launchpad-dev 2010-08-05
<lifeless> you want to file the bug, or shall I ?
<lifeless> the lower our latency the more often this will happen
<james_w> wgrant: all in ec2
<lifeless> does anyone else see  lp.registry.windmill.tests.test_person_picker.TesPersonPickerWidget.test_person_picker_widget failing ?
<lifeless> WindmillTestClientException: {u'suite_name': u'PersonPickerWidget', u'result': False, u'starttime': u'2010-7-4T10:26:38.547Z', u'params': {u'xpath': u'//div[contains(@class, "yui-picker ") and not(contains(@class, "yui-picker-hidden"))]', u'timeout': u'20000'}, u'output': None, u'endtime': u'2010-7-4T10:26:59.337Z', u'method': u'waits.forElement'}
<lifeless> branch is a few days old
<thumper> lifeless: running locally or through ec2?
<lifeless> local
<thumper> I'll try it here
<thumper> is it really TesPersonPickerWidget, not Test... ?
<lifeless> lp.registry.windmill.tests.test_person_picker.TesPersonPickerWidget.test_person_picker_widget
<lifeless> I have a few other failures
<lifeless> lp.registry.browser.tests.test_person_view.TestPersonRelatedSoftwareView.test_latest_uploaded_ppa_packages_with_stats
<lifeless> lp.registry.browser.tests.test_person_view.TestPersonEditView.test_can_rename_with_deleted_PPA
<lifeless> and similar
<thumper> lifeless: works for me
<lifeless> thanks
<wgrant> james_w: Thanks.
<wgrant> lifeless: Looks like we can destroy PublishedPackage easily.
<lifeless> wgrant: \o/
<lifeless> wgrant: Nuke It, Nuke It, Nuke It
<wgrant> Particularly if we don't care about create-debwatches.py, which probably doesn't work anyway.
<lifeless> what doth that do ?
<wgrant> It imports Debian's bugs into LP.
<wgrant> But it might be easy enough to port.
<wgrant> It's never been used, and is several years old.
<lifeless> harh
<lifeless> so, does it have tests ?
<lifeless> actually let me rephrase.
<lifeless> does it have any reason to be in the LP core rather than an API client ?
<wgrant> It probably impersonates.
<lifeless> ok, so we need impersonation in the API eventually
<lifeless> but not now
<wgrant> There are no tests that I can see. So I'll just port it and pretend that it works.
<lifeless> \o/
<lifeless> thank you _very_ much for doing this
<wgrant> The other PublishedPackage callsite only needs ~6 of the 13 tables.
<lifeless> mars: still up ?
<wgrant> So I'll replace it with another method, and drop the view.
<wgrant> And hopefully get it landed tonight.
 * mwhudson gets ready to run some windmill tests
<lifeless> awesome!
<mwhudson> i wonder if i can get to the shop and back before they're done
<lifeless> mwhudson: 'yes'
<mwhudson> i guess everything will time out
<lifeless> spm: hey, so, where on your queue is the +icing rollout tweak ?
<spm> lifeless: it's not on mine tbh; still working on the next phase of the slony1 roll; getting some LS stuff done; and some 'fit in the cracks' U1 more trivialish scripting stuff. and packing at some point.
<lifeless> spm: ok; you're sprinting or something ?
<spm> yup all next week
<lifeless> cool
<lifeless> do you guys deliberately do these on release weeks ?
<lifeless> A;)
<spm> ha!
<spm> the release we did for LP in barcelona last year was ... instructive for the rather large collection of observers and sympathisers I felt.
<spm> probably freaked the hotel staff out a tad I'll bet :-)
<spm> 15 folks all hanging aournd 2-3 laptops....
<lifeless> spm: if you have the opportunity to do the icing thing, and wanted a bribe... well, just let me know :)
<mwhudson> yes, my machine is too slow to run the windmill tests
<mwhudson> can i get someone to run some tests for me?
<lifeless> mwhudson: yes; you could use ec2 though, FWIW
<mwhudson> yes, that's true
<mwhudson> lifeless: could you get
<mwhudson> bzr+ssh://bazaar.launchpad.net/~mwhudson/launchpad/vostok-main-template
<mwhudson> and run
<mwhudson> lp.code.windmill.tests.test_branchmergeproposal_commitmessage.TestCommitMessage.test_set_commit_message
<mwhudson> lp.code.windmill.tests.test_branchmergeproposal_review.TestReviewCommenting.test_merge_proposal_replying
<mwhudson> ?
<spm> lifeless: we had a brief discussion some weeks back in losa land around some topics we could present at various other team sprints; i'll give you the brief synopsis of each one? it may help with your previous question?
<spm> * "Why Cake is Good For Keeping LOSAs Onside" - Well. Dur.
<spm> * "Let Them Eat Cake" - a discussion around various types of Cakes enjoyed by the losas, accounting for cultural variations across the team. Mudcakes, Cheesecakes, Pavlova, Poutine Cake etc
<spm> * "Haz Cakez, Willz Patchz Youz Servicez" - we are open to bribery to apportion RT priorities more ... appropriately. A look at proposed cake:priority bribe ratios.
<spm> * "Not Cake" - a free flowing discussion around other non cake foods - eg brownies, fudge; and how they may be treated as an acceptable substitute when Cake is not as easily forthcoming.
<spm> lifeless: ^^
<lifeless> rotfl
<mwhudson> poutine cake?
<lifeless> spm: so, courier, or pickup from the bakery?
<spm> we just come back from MontrÃ©al, poutine cake seemed appropriate at the time.
<lifeless> mwhudson: does it need a make schema ?
<mwhudson> lifeless: no
<lifeless> :!testr run -- --load-list mwhudson.list
<lifeless> please wait ;)
<mwhudson> hm, it seems i didn't have the wadl built locally, does that affect using the api over js?
<lifeless> ni idea
<poolie> speaking of speed, is it really recessary to spend 15s rebuilding js at the start of 'make run'?
<lifeless> windmill.server.proxy: INFO     Could not fullfill proxy request to http://www.google.co.nz/+icing/rev10721/MochiKit.js
<lifeless>  ?
<lifeless> mwhudson: test_results: ERROR    Test Failure in test {'version': '0.1', 'suite_name': u'Commit message editing.', 'result': False, 'starttime': u'2010-7-5T6:8:14.422Z', 'output': None, u'params': {u'xpath': u'//div[@id="edit-commit_message"]//textarea[@class="yui-ieditor-input"][1]', 'uuid': 'b99e4c88-a029-11df-8e6c-5254002410b3'}, 'endtime': u'2010-7-5T6:8:27.487Z', u'method': u'waits.forElement'}
<wgrant> mwhudson: It doesn't, AFAIK.
<lifeless> id: 197 tests: 22 failures: 1 skips: 1
<mwhudson> lifeless: ok thanks
<lifeless> mwhudson: that google.co.nz error worries me :)
<mwhudson> yes, does look pretty dodgy
<mwhudson> lifeless: can you run it in trunk too?
<lifeless> mwhudson: what do you mean?
<mwhudson> lifeless: does the test fail in trunk for you too?
<lifeless> devel or db-devel
<mwhudson> lifeless: devel
<lifeless> trying
<mwhudson> thanks
<lifeless> no
<lifeless> erm
<lifeless> yes
<lifeless> yes it fails
<mwhudson> :(
<mwhudson> well i did what the test does by hand in a real browser and it worked, so...
<wgrant> lifeless: Since those two are the same bug (Distribution.guessPackageNames is slow), should I dupe them and move to launchpad-registry?
<lifeless> wgrant: sure
<lifeless> wgrant: though I would have said it was a soyuz problem :P
<wgrant> Although both bugs seem to cover other issues too..
<lifeless> wgrant: do what you think makes the most sense.
<lifeless> make a third bug even.
 * wgrant does so.
<wgrant> Oh look, PublishedPackage is now unused.
 * wgrant drops.
<lifeless> \o/
<wgrant> It looks like the view's main user was removed for 3.0.
<wgrant> The distribution package listing was eliminated.
<lifeless> seeking feedback on https://help.launchpad.net/Oops
<mwhudson> lifeless: looks pretty good to me
<lifeless> mwhudson: care do to a small review for me? see #lp-r
<james_w> mwhudson: if you are having trouble running problematic tests then feel free to drop me an email and I can take a look tomorrow
<mwhudson> james_w: i might do that thanks
<wgrant> I have a slave query returning an object that gets passed around and eventually dies when some code tries to link it to a master object.
<wgrant> Where's the right place to do the IMasterObject adaption?
<mwhudson> i think it's basically impossible to say on this much information :-)
<mwhudson> i guess in the linking place, maybe?
<wgrant> Yeah, but it's a reasonably complex stack, so I decided to omit the messy description.
<wgrant> Some Bugs views use Distribution.guessPackageNames to retrieve a sourcepackagename and binarypackagename.
<wgrant> I've just moved that query to a slave.
<wgrant> Eventually the view calls Bug.addTask with one of those two slave objects.
<wgrant> This then tries to create a DistributionSourcePackageInDatabase, which crashes with a WrongStoreError.
<mwhudson> wgrant: i guess addTask would be a reasonable place?
<wgrant> Ah, hm. The problem is actually when it generates the DistributionSourcePackage (not the DistributionSourcePackageInDatabase). I guess Distribution.getSourcePackage should make sure that its sourcepackagename is from the right store.
<wgrant> I wonder if there's an easy way to do that (get an object from the store of another object).
<wgrant> Nyeh. i guess I'll just IMasterObject it before I call getSourcePackage.
<lifeless> whats an IMO ?
<mwhudson> lifeless: it's a thing that lets you ensure that a database object is from the master store
<mwhudson> master_object = IMasterObject(master_or_slave_object)
 * mwhudson afk briefly
<wgrant> Argh
 * wgrant stabs canonical.widgets.
<wgrant> Nasty code, evading my grep.
<spm> wgrant: that sounds suspiciously like the Bad Workman Blaming The Tools???
<lifeless> this from a sysadmin?
<lifeless> :P
<spm> point, veritable good point. 1/0
<wgrant> spm: Well, I expect stuff to live in lp.*, or canonical.launchpad.* at worst...
<spm> :-)
<lifeless> wgrant: bzr-grep ?
<wgrant> lifeless: grepping the whole LP tree on a cold cache is slow.
<lifeless> wgrant: :<
<wgrant> lifeless: Can I have a patch number for the PublishedPackage drop?
<mwhudson> wgrant: grepping everything can't be much slower than grepping lib/lp and lib/canonical/launchpad surely?
<mwhudson> at least if you use bzr-grep or bzr ls | xargs
<mwhudson> grepping all the eggs and sourcecode does take a while
<wgrant> mwhudson: Right, eggs and sourcecode takes a while.
<wgrant> I should probably install bzr-grep.
<lifeless> wgrant: except when stubs on leave, he should be the one handing them out
<lifeless> wgrant: it is a little more latency, but I think its better to have no confusion in this area ;)
<wgrant> Ah, so a longer-term definition of 'not available' than I thought.
<wgrant> OK.
<lifeless> well, thats my understanding
<wgrant> yeah, you're probably right.
<lifeless> I'll check with him that that is our shared understanding, and tweak the wiki page to be clearer one way or another
<wgrant> I was just remembering the email from a couple of days ago.
<mwhudson> someone could write a webservice to hand out patch numbers!
 * mwhudson runs away
<lifeless> mwhudson: slap!
<lifeless> the lp login pages to the list is really quite entertaining
<lifeless> s/lp/sso/
<mwhudson> or stub and lifeless could use some kind of distributed voting algorithm
<lifeless> we do
<lifeless> he votes
<lifeless> I don't.
<lifeless> see, distributed.
<wgrant> lifeless: Which login pages?
<mwhudson> wgrant: i typed "bzr ls --versioned --recursive --null --kind file | xargs -0 grep" once and have a super huge bash_history file :-)
<wgrant> mwhudson: Heh.
<mwhudson> lifeless: :-)
<lifeless> wgrant: the thingy ones
<lifeless> wgrant: team oops summaries
<wgrant> lifeless: Ah.
<mwhudson> wgrant: we get oops summaries to the internal mailing list
<wgrant> Right.
<lifeless> and they are baaaagggered
<mwhudson> except for the last couple of days, the bodies of the mails have been openid login pages
<wgrant> hahaha.
<wgrant> Nice.
<mwhudson> i don't even really understand how this is possible
<wgrant> It certainly leaves me concerned as to how they are generated...
<lifeless> mwhudson: cron. Datacentre. \o/
<lifeless> wgrant: the main oops reports work ok, its the team summaries that are being pulled from a web service that are booming
<wgrant> mwhudson: I'm guessing you have a PQM rejection for my branch?
<wgrant> It probably conflicted with james_w.
<wgrant> Yes :(
<mwhudson> wgrant: mail.canonical.com is being administered, so i don't have any mail at the moment
<wgrant> 'administered'. Nice euphemism.
<wgrant> Hm.
<wgrant> I wonder if lack of MTA means the three other branches that have just succeeded will fail to submit...
<mwhudson> well
<mwhudson> at worst they'll just get queued
<wgrant> Hopefully.
<wgrant> I doubt it, though.
<mwhudson> wgrant: it's planned maintenence
<wgrant> mwhudson: But doesn't pqm-submit use mail.canonical.com:25, which is currently timing out?
<wgrant> Oh, maybe smtp.canonical.com, which works.
<mwhudson> yeah
<mwhudson> mail.canonical.com:25 gets eaten by the firewall i imagine :-)
<lifeless> anyone got a few minutes to point me at a good example of a pyunit browser test ?
<spiv> mwhudson: thanks for landing the codebrowse-oops branch!
<mwhudson> spiv: np, have you looked at lp-production-configs yet?
<spiv> no, I was about to ask about the production config stuff
<spiv> I'd be happiest if it was just SEP ;)
<lifeless> spiv: mail losas@c.c asking for the config details to use
<lifeless> then edit the lp-production-configs branch to make the change, and propose a merge
<spiv> lifeless: well, more specifically, I was asking if mwhudson had taken care of that already, as he'd landed the branch
<lifeless> spiv: I know that that would make you happiest :)
<mwhudson> spiv: i haven't done anything in that regard
<lifeless> wgrant: hi
<spiv> mwhudson: ok.  I might look into that tomorrow then.
<thumper> wasn't there going to be an email outage?
<mwhudson> thumper: only if you're an imap user maybe?
<wgrant> lifeless: Fail. I forgot to add the DB patch.
<lifeless> wgrant: so does my suggestion make sense
<lifeless> wgrant: recommit your branch against devel
<lifeless> do *just* the db-patch against db-devel
<lifeless> and do that after the other branch has landed
<wgrant> lifeless: Hm, I suppose, but we freeze in 24 hours so there's no point.
<lifeless> wgrant: oh, fair point.
<lifeless> anywhich way is good for me
<wgrant> lifeless: DB patch pushed.
<thumper> StevenK: ping
<lifeless> wgrant: url ?
<wgrant> lifeless: https://code.edge.launchpad.net/~wgrant/launchpad/destroy-publishedpackage/+merge/31813
<lifeless> spm: An updated diff will be available in a few minutes. Reload to see the changes.
<lifeless> spm: after 10 minutes...
<mwhudson> uhm
<mwhudson> is merge-proposal-jobs stuck again?
<thumper> I hope not
<thumper> it is possible it is a big job...
<thumper> maybe
<thumper> spm: sync logz plz?
<lifeless> thumper: mwhudson: does code have pyunit based browser using tests ?
<lifeless> not windmill, the zope mechanise thingy
<mwhudson> lifeless: yes
<wgrant> zope.testbrowser?
<mwhudson> lifeless: grep for getUserBrowser iirc
<lifeless> mwhudson: please point me at some as per my plea for help above ;P
<wgrant> lp.code.browser.tests looks relevant.
<lifeless> mwhudson: thanks
<thumper> BrowserTestCase in lp.testing I believe
 * thumper is smacking his head against inmemory xmlrpc implementation
<spm> thumper: syncing in progress
<thumper> spm: ta
<thumper> spm: merge proposal jobs is stuck again
<thumper> spm: plz kill
<spm> killed
<spm> logs resynced
<thumper> ta
<spm> thumper: looks like a couple of bodgy branches from that shyster wgrant again. any chance of you and me flying down to his place and ... trying some attitude adjustment? :-P
<wgrant> Heh.
<thumper> spm: you book the flights, I'll approve them ;-)
<spm> oki, brb. one sec.
<spm> wgrant: so that destroy-publisher one was first off - should be good now
<wgrant> spm: destroy-publishedpackage? Sounds good.
<wgrant> I hope I'm not destroying the publisher itself!
<spm> wgrant: destroy-publishedpackage yup. too lazy to select and paste :-)
<wgrant> lifeless: diff's there now.
<spm> lifeless: fwiw, that's bug https://bugs.launchpad.net/bugs/605772 (tongue firmly in cheek) you may be able to get the priority on a fix raised?
<_mup_> Bug #605772: merge-proposal-jobs is "hanging", apparently with nothing to do <canonical-losa-lp> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/605772>
<j24> did someone known whereis the admin interface of launchpad?
<mwhudson> if that's the same bug as was hanging the branch mirror script once upon a time, it's a complete mystery
<spm> j24: it doesn't really have one. some pages have extra features if you have admin access is about it
<j24> how can I remove projects?
<j24> how can I manage user right?
<thumper> j24: set them inactive
<thumper> j24: what type of user rights?
<j24> thumper: read write acces to projects
<spm> create a team; give that team project ownership; add them to the team
<thumper> j24: the owner needs to be set to a team that the users are members of
<wgrant> j24: Have you used Launchpad before?
<j24> never
<wgrant> You probably shouldn't be trying to run your own if you haven't.
<j24> I'm looking for a solutoin to manage bazar and hear that launchpad will the good one
<j24> for now we use redmine
<j24> I'm evalluating bazar as a replacement
<j24> I'm evalluating launchpad as a replacement
<lifeless> spm: asking for a backtrace
<j24> patch-2207-51-0.sql has not been applied to the database. You probably want to run 'make schema'.
<lifeless> spm: it may help shed light, though the unhandled error is a pretty good hint
<spm> lifeless: some more context?
<j24> but when I did sudo -u postgres make create
<j24> the patch is reported apply
<lifeless> spm: 15:57 < spm> lifeless: fwiw, that's bug https://bugs.launchpad.net/bugs/605772 (tongue firmly in cheek) you may be able to get the priority on a fix raised?
<_mup_> Bug #605772: merge-proposal-jobs is "hanging", apparently with nothing to do <canonical-losa-lp> <Launchpad Bazaar Integration:Incomplete> <https://launchpad.net/bugs/605772>
<j24> any idea?
<spm> lifeless: Oh right.
<lifeless> j24: well, launchpad is *intended* to be a hosted solution, so the easiest thing you can do use use launchpad.net. However I realise that isn't answer your question, so let me have a guess
<lifeless> j24: you appear to be running make create, not make schema.
<lifeless> j24: the 'make' targets are meant for developers, not for production rollouts / upgrades
<lifeless> j24: so, probably, you need to run 'make schema'
<j24> lifeless: I did first
<j24> lifeless: but I nned to run the create as postgres
<j24> use
<j24> r
<wgrant> mwhudson: Could you resubmit https://code.edge.launchpad.net/~wgrant/launchpad/per-archive-build-debug-symbols/+merge/29671, please? The conflict is fixed.
<j24> lifeless: is ther any easy way to create your own host of launchpad?
<wgrant> No.
<mwhudson> wgrant: ok
<wgrant> mwhudson: Thanks.
<lifeless> j24: so, If you have run 'make schema' (*not* make create), as postgres, and you are now running the appserver as a different user, I suspect postgres permissions will be stopping it reading the db properly
<lifeless> hmmm, byebye wave
<wgrant> lifeless: Well, they don't say it's being shut down.
<wgrant> Just that they're ceasing development...
<lifeless> they say end of year shutdown
<wgrant> Oh?
<wgrant> Didn't see that.
<j24> lifeless: I've to run make schema as potgress to fix it
<j24> lifeless: how can I've a empty database ?
<j24> lifeless: no projects
<wgrant> lifeless: Ah, there it is.
<lifeless> wgrant: its not quite 'turn off', but the option is well implied
<wgrant> lifeless: And thanks for the review.
<lifeless> wgrant: de nada
<wgrant> [A2
<lifeless> +1
<wgrant> lifeless: Serious WTFery at that link in #launchpad.
<wgrant> Looks like the TAL caching being strange...
<wgrant> He sees three of one milestone, while I see two of each.
 * ajmitch sees nothing wrong
<wgrant> ajmitch: Refresh a couple of times.
<wgrant> Or it may require the addition of some more.
<wgrant> Aha, yes.
<wgrant> I initially viewed it when there were only three.
<wgrant> Now there are six, and I see the first three repeated twice.
<wgrant> He initially viewed it when there was one.
<wgrant> Then added two more, and saw three of the first one.
<thumper> wgrant: what is the A reference?
<wgrant> thumper: Hm?
<thumper> <wgrant> [A2
<lifeless> so, thats the milestone caching I would like to disable
<lifeless> as one of the examples of we're-not-ready-yet memcached use
<wgrant> thumper: That was some network lag converting my up arrow (which would normally revive the previous irssi command) into some garbage.
<wgrant> Not sure why it does that.
<wgrant> lifeless: No, this is a bug.
<thumper> ah
<wgrant> lifeless: Something's wrong in the caching infrastructure.
<lifeless> wgrant: tell me more
<wgrant> lifeless: It's using the cache for the wrong object, somehow.
<wgrant> Hmm, interesting.
<wgrant> Ah, got it.
<lifeless> ?
<wgrant> Well, not entirely, actually, as it turns out.
<wgrant> But the way it handles tal:cache in loops seems a bit odd.
<thumper> :(
<thumper> grr
<wgrant> Hm?
<lifeless> I need a another test hint
<lifeless> I want to exercise an API
<lifeless> is doing a single browser request to the relative URL the OOPS shows sufficient ?
<lifeless> e.g. https://api.launchpad.net/beta/~ubuntu-dev/participants
<lifeless> or is there a better way ?
<lifeless> my goal is:
<lifeless>  - exercise the web request that is blowing up
<lifeless>  - measure its query count
<lifeless>  - fix it
<mwhudson> lifeless: there's something called a WebserviceCaller somewhere i think
<lifeless> thanks
<lifeless> LaunchpadWebServiceCaller
<wgrant> lifeless: So, the TAL memcached stuff adds the loop index to the key.
<wgrant> lifeless: What goes wrong is that the indexes change...
<wgrant> I'm not sure whether this is a Registry or Foundations bug.
<lifeless> so the key is host/object/template/loopN (or something)
<lifeless> uhm
<wgrant> Pretty much, yes.
<lifeless> yes, clearly thats a problem
<lifeless> cache at canonical url only basically
<lifeless> mwhudson: newInteraction called while another interaction is active.
<lifeless> mwhudson: I have a feeling you know about this
<lifeless> mwhudson: http://pastebin.com/pkBkT68i
 * mwhudson clicks, waits
<mwhudson> lifeless: has pastebin.com fallen over?
<wgrant> lifeless: Log out before you log in again.
<mwhudson> ah there it is
<StevenK> http://downforeveryoneorjustme.com/pastebin.comhttp://downforeveryoneorjustme.com/pastebin.com
<StevenK> :-P
<mwhudson> lifeless: yes, you need to logout before doing web requesty type stuff
<mwhudson> TestCaseWithFactory.setUp() logs in as anonymous
<mwhudson> lifeless: all of this is fairly horrible
<lifeless> oh, it _fail_
<lifeless> how unexpected
<lifeless> how much would die if that got fixed?
<mwhudson> i'm not even sure what a fix would be like
<lifeless> don't log in as anonymous
<mwhudson> oh right
<mwhudson> well, dunno
<mwhudson> probably plenty
<lifeless>   File "/home/robertc/launchpad/lp-branches/working/lib/lp/testing/factory.py", line 434, in makePerson
<lifeless>     person, email = getUtility(IPersonSet).createPersonAndEmail(
<lifeless> like that :P
<lifeless> its odd that anonymous can createPersonAndEmail, or is that just me ?
<mwhudson> lifeless: there's a special case somewhere
<lifeless> *blink*
<mwhudson> you have to be able to create person records without being logged in ...
<stub> Is anyone landing https://code.edge.launchpad.net/~wgrant/launchpad/per-archive-build-debug-symbols/+merge/29671 ?
<mwhudson> i think i'm trying to
<stub> k
<mwhudson> yes
<mwhudson> it's in ec2
<wgrant> stub: It was rejected by PQM a few hours ago due to a conflict, but should land in a couple of hours.
<lifeless> man
<lifeless> its very surprising to have 'url = "/~%s/participants" % team.name' fail if you don't have something logged in.
<lifeless> _not python anymore_
<lifeless> garh
<lifeless> and now
<lifeless> response 'Anonymous requests must provide a User-Agent.
<lifeless> '401' ><
<wgrant> lifeless: You know there are a couple of webservice wrapper utilities, right?
<wgrant> There's the old one, and then there's launchpadlib.
<mwhudson> haha, after 5 or so days, my recipe build fails with a stupid typo
<lifeless> wgrant: I'm using LaunchpadWebServiceCaller
<lifeless> wgrant: I want to do exactly one http request, and launchpadlib is very not-geared-to-control-that.
<wgrant> lifeless: Shh.
<lifeless> wgrant: its peanut butter jelly time
<wgrant> stub: Thanks.
 * mwhudson creates a mercurial import, optimistically
<lifeless> bbiab
<mwhudson> wow, it completed in 60 secs
<wgrant> mwhudson: Successfully?
<mwhudson> wgrant: yeah
<wgrant> Hm.
<wgrant> Did it have any revisions?
<mwhudson> 662 apparently
<adeuring> good morning
<lifeless> I am in a maze of unfinished toolchains
<wgrant> Oh?
<lifeless> wgrant: just building up my normal test flexability
<wgrant> Ah.
<lifeless> the next thing will be make_query_checker(LessThan(20))
<wgrant> Excellent idea.
<lifeless> part 1 is up for testtools review already
<lifeless> jml: hi
<lifeless> testtools trunk has UTF8_TEXT in news and PLAIN_TEXT in content_type
<mrevell> Hello
<lifeless> mrevell: hello
<thumper> does loggerhead stay running for anyone else using `make run_codehosting`?
<lifeless> james_w: btw, have you seen testtools.matchers.Annotate ?
<poolie> lifeless: what does that do?
<lifeless> adds : LITERAL to the mismatch
<lifeless> which some of james new code in lp.testing.matchers does as well
<poolie> nice
<lifeless> jml: hi, awake?
<hrw> hi
<wgrant> bigjools: Hi. lamont's fine with https://code.edge.launchpad.net/~wgrant/launchpad/multi-arch-builders/+merge/31740 -- how 'bout you?
<lifeless> wgrant: http://pastebin.com/B72J7fuA
<wgrant> lifeless: Hmm?
<lifeless> thats what the test infrastructure I just finished putting together reports
<wgrant> lifeless: Oh, right. I just looked at the two tracebacks at the top and wondered what I'd broken :P
<lifeless> lp.registry.tests.test_person.TestAPIPartipication.test_participation_query_limit is the only interesting bit
<wgrant> Yep.
<wgrant> Looks good.
<jml> lifeless, am now.
<stub> Monster storm coming through. If I'm gone its a power outage.
<lifeless> jml: hi
<lifeless> jml: i added another matcher
<jml> lifeless, approved
<lifeless> jml: and found a glitch in testtools trunk - UTF8_TEXT doesn't exist, but NEWS claims it does.
<jml> lifeless, oh right. easily fixed.
<jml> although a shame that glitch is in a release
<lifeless> jml: I haven't uploaded the release to debian
<lifeless> cause I've been lazy
<jml> http://pastebin.ubuntu.com/473448/
<lifeless> jml: +1
<lifeless> jml: you might like https://code.edge.launchpad.net/~lifeless/launchpad/registry/+merge/31830
<jml> Python's warnings filter doesn't work the way I expected.
<jml> lifeless, nice.
<jml> oh, this might not be Python
<jml> le sigh
<lifeless> ?
<jml> https://code.edge.launchpad.net/~jml/launchpad/show-warnings-once/+merge/31831
<jml> heh. lifeless, I've got five branches that you alone can review.
<jml> jelmer, I've got one branch that you can review: https://code.edge.launchpad.net/~jml/subunit/move-ls/+merge/31769
<lifeless> jml: testrepository ones?
<jml> lifeless, yeah.
<lifeless> I'm sorry.
<lifeless> I suck
<lifeless> I want to review them, honest.
<jml> that's ok.
<jml> each of them is a usability improvement (imo), so keen to get them landed :)
<jml> then all I need to do is add native --load-list support to zope.testing and trial and then that branch of depth-first traversal of my hacking problems will be finished
<jelmer> jml: I'll see if I can do a review during an evening this week.
<jml> jelmer, thanks. it really is a very simple change.
<jelmer> ah, ok - I haven't looked yet
<lifeless> jml: cool.
<lifeless> jml: I'm still bootstrapping myself to proper effectiveness in lp's code base
<jml> lifeless, me too!
<lifeless> jml: har har har
<lifeless> jml: I'm fairly sure you know where all the skeletons are by now :)
<lifeless> later
<jelmer> jml: +1 on the user side of your MPs, though I wonder if it's always a good idea to initialize on load if there's no .testrepository directory.
<jml> jelmer, what possible harm could there be in doing so?
<jelmer> e.g. me running "bzr selftest --subunit | testr load" but in the wrong directory
<jml> I guess the harm there is you "lose" a test run to another repo & don't know about it
<jml> vs the harm of pissing me off every single time I make a new branch.
<lifeless> jml: fwiw testr init returns 0 if there is already a repo
<lifeless> jml: to allow testr init; testr run/load/whatever
<lifeless> jml: I did this to permit both styles of work
<lifeless> I think.
<lifeless> ETired. really gone.
<jml> lifeless, ok, good night.
<deryck> Morning, all.
<jml> deryck, good morning.
<deryck> hi jml.  Do you have some time tomorrow that you and I could chat?  Just to catch up a bit for one, and then I have some LEP and LEP-like things to chat about.
<jml> deryck, sure.
<jml> lemme consult my calendar
<jml> deryck, how's 1400UTC?
<deryck> jml, perfect.  Thanks!
<deryck> heh, we double calendared.  I'll accept yours, jml.
<jml> deryck, ta.
<jml> this wireless card dropping thing is amazingly disruptive. I might as well have my computer spontaneously reboot on me.
<mars> wgrant, ping, got a failure running ec2 land last night: lib/lp/archiveuploader/tests/nascentupload-ddebs.tx
<wgrant> mars: Yeah, fixed and landed already :)
<wgrant> Thanks.
<mars> wgrant, ok
<mars> np
<jtv> danilos, henninge: I filed bug 613821 for the approver failure, and an incidental bug 613823
<_mup_> Bug #613821: Approver violates unique constraint <Launchpad Translations:New> <https://launchpad.net/bugs/613821>
<_mup_> Bug #613823: Approver should log oopses <Launchpad Translations:New> <https://launchpad.net/bugs/613823>
<danilos> jtv, thanks, 613923 is not very useful unless we plan to start working on it, and we don't
<jtv> danilos: well I admit fixing it would be bad from the zero-oops perspectiveâ¦
<danilos> jtv, (because the same holds for language-packs, po exports, pofile stats,...)
<jtv> OTOH it's basically saying we're accepting oopses as a matter of course.
<danilos> jtv, no, it's just that it's a known problem with most of our old scripts
<jtv> Definitely.  But in this case, it's hiding oopses.
<danilos> jtv, it's not, that's why we have it logging with full detail
<jtv> danilos: do you check those logs regularly?
<danilos> jtv, OOPSes are hidden for those parts where we have to dig through a million-OOPS count, this is all in easy to look at log files
<danilos> jtv, yes
<jtv> Okay, then this falls under "time-saver" not under "oopses."
<jtv> How long had we been having that approval conflict I posted about?
<danilos> jtv, well, not really, since at the moment oops tools could only provide us with *another* email instead of aggregating it into one
<danilos> jtv, we've had it appear and go for a few days already
<danilos> jtv, you were mostly tracking them though
<danilos> jtv, or maybe I am talking about a different approver bug
<jtv> I don't see how the approval conflict could go away.  We had a different traceback a few days ago, for which I cleaned up the data.  And now we're having a different kind of traceback.  But as a third problem, there's the KDE approval conflict where it's been logging that there were two templates with the same domain.
<jtv> I hope that will stay as unusual as I find it now: we find 3 completely different approval problems in 1 week.
<jtv> grrr staging buildfarm hurry up
<danilos> jtv, yeah, this one was hitting us for 3 days now
<jtv> danilos: 3 days is about the maximum time for this approver problem.  If the older entry is still in the way after that time, chances are it's in Failed state and will stay around for much longer.  :(
 * jtv updates bug report with this note
<stub> What is our preferred mock logger?
<deryck> mrevell, ping
<mars> stub, for launchpad?  I've used mocker elsewhere, but not for logging.
<mrevell> Hi there deryck
<stub> There are three or four mock loggers for tests in the tree I believe, and I can't find any of them (including the one I wrote)
<mars> abentley, ping, any idea why this project /still/ insists that there is no default branch, even though the website says I did? https://code.edge.launchpad.net/~launchpad-qa/qa-shepherd/devel
<abentley> mars, default for what?
<mars> abentley, for this qa-shepherd project
<mars> sorry, default for the project
<abentley> mars, do you mean "development focus"?
<stub> lp.testing.logger defines MockLogger
<mars> abentley, no idea.  All I know is that I tried what our website says, "bzr branch lp:qa-shepherd", and bzr returns an error saying "bzr: ERROR: Invalid url supplied to transport: "lp:qa-shepherd": qa-shepherd has no default branch."
<mars> abentley, here is the project page: https://edge.launchpad.net/qa-shepherd
<mars> abentley, everything looks OK to me on that page
<mars> stub, then I would take the fact you can't find the other three instances as a good thing :)
<stub> There is a comment from 2007 in there pointing to one of the others (launchpad.scripts.logger.FakeLogger)
<stub> Maybe it is a daisy chain - I don't want to look.
<mars> hehe
<abentley> mars, no, I've never heard of a problem like this before.
<mars> abentley, it sounds like a branch policy violation.  I wonder, could the branch policy for the project still be 'private by default' or something?
<mars> abentley, hmm: https://code.edge.launchpad.net/qa-shepherd - the development focus is stacked on my branch, and my branch is private
<abentley> mars, I would not expect the branch policy to matter.  I would expect only the branch privacy to matter.
<mars> could that do it?
<abentley> mars, that sounds possible.
<mars> abentley, and it works now.  Shall I file a bug?
<abentley> mars, please do.
<abentley> mars, did you do anything to fix it?
<mars> abentley, I changed my branch (the stacking base) to a public branch
<mars> from a private one
<mars> that allowed the project branch, which was stacked on top of mine, to start working
<abentley> mars, so the bug here is that you shouldn't be able to set a branch public without also setting its stacked-on branch public.
<mars> right
<abentley> Probably some better diagnostics would be good too.
<bigjools> jml: I've been trying to break the new buildd-manager in dogfood and it's coped with everything I can throw at it so far.  I'm shocked.  I even saw it resetting one builder at the same time as dispatching to another, which is a first.
<bigjools> time to land it I think
<mars> abentley, bug filed: 613841
 * jelmer wonders why dev.launchpad.net doesn't let him subscribe to pages
<abentley> mars, we only use the development focus branch for stacking on.
<abentley> mars, so your recipe implies "2.5 make lp:~me/theproject/devel the development focus of the project"
<mars> jelmer, our custom Moin theme dropped a few wiki features, like subscribing to pages and smilies.  I have no idea why though.
<bigjools> jelmer: you have to url hack
<bigjools> jelmer: ?action=subscribe
<mars> beat me to it
<jelmer> bigjools, mars: thanks!
<mars> abentley, you are right, that is what I did, yet
<mars> yes even!
<mars> Ah, text searching - that feature was dropped too
<mars> So you have to do a title search, then click the 'do a text search' button on the title search results page
<jml> bigjools, cool :)
<bigjools> jml: but given that we have nothing like a production environment to test with, I'm still scared :)
<bigjools> so I just did a full test run for the first time in a while, and I'm astonished at the fact that the output is basically doubled in size with all the "warnings" at the end
<jml> bigjools, yes. you may have noticed some complaints about this on the list :)
<bigjools> jml: complaints? really? :)
<mars> gary_poster, ping, I have a buildout question for you when you have a moment
<gary_poster> mars, ack, on call.  will ping when off
<mars> ok
<sinzui> mrevell, ping
<mrevell> yo sinzui
<bigjools> sinzui: so, where are distroseries.title, displayname, summary and description all used?  I want to fix the crappy descriptions on the addseries page.
<sinzui> bigjools, I think they are used on ds+index, ds+series, milestone+index. I think you need to grep for series\.(title|displayname|summary)
<bigjools> sinzui: yeah I did, I can't work out why we have a separate summary and description
<bigjools> do you know of any hysterical raisins?
<sinzui> bigjools, There is also an calculated attr (named_version) That is more often used. I regret creating it. I think the lts/name/version issue could have been solved if I really made displayname calculated
<sinzui> bigjools, summary may only be useful on +series
<bigjools> sinzui: yes, the difference between Display Name and Title eludes me a bit too
<sinzui> bigjools, I also believe that mpt was right in suggesting we abandon title
<bigjools> yeah, it seems redundant
<bigjools> perhaps now is the time, while I am redesigning the addseries page
<sinzui> Yes
<sinzui> I think so
<bigjools> ok, I'll exclude it from the mockups
<mtaylor> sinzui: hey - don't mean to be a pest, but any luck figuring out why that guy isn't getting mailing list mail? he's starting to get frustrated
<sinzui> I have no power in that regard. I asked an admin to look at the error and vette log for evidence of problems
<mtaylor> sinzui: excellent. thanks!
<sinzui> mtaylor, when staging gets a copy of production data in 3 days, I will have some power to see what has happened. So if and Admin has not solved the issue, I can at least look for problems
<mtaylor> sinzui: cool. don't you love odd problems like this one? :)
<Ursinha> sinzui, hello. do you know what is a ProfilingOops? https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1676S100
<sinzui> Ursinha, lp.services.profile. lifeless created it recently
<Ursinha> aha, I was suspecting
<sinzui> Ursinha, is is manually enable-able on staging to do research into timeouts. This looks like it did not work
<Ursinha> it seems
<sinzui> oh
<Ursinha> I see ~200
<Ursinha> I also see a lot of DisconnectionErrors...
<sinzui> profiling is one and it is for everything. then you turn it off. you are trying to get exclusive use of staging to do a test. But...this example shows the xmlrpc polling by mailman was running during the test
<sinzui> s/one/on/
<Ursinha> I see
<Ursinha> sinzui, about the accountmerge timeouts... is there a bug for that? I only found one bug, a very old one
<Ursinha> and doesn't seem related
<sinzui> Account merge (profile merge) is 40 steps. the oopses can mutate. The same code is also used for person and team, for user requested and admin requested. one code, 160 kinds of oopes
<sinzui> Ursinha, we agree we cannot fix the timeout by changing the code shown in the oops we need a new process.
<sinzui> I created a bug tag to help see the problem
<Ursinha> sinzui, right
<sinzui> Ursinha, https://bugs.edge.launchpad.net/launchpad-registry/+bugs?field.tag=merge-deactivate, The real problem will be fixed with bug 162510
<_mup_> Bug #162510: Person merging needs to be done asynchronously <chr> <feature> <merge-deactivate> <tech-debt> <timeout> <Launchpad Registry:Triaged> <https://launchpad.net/bugs/162510>
<Ursinha> looking
<deryck> hurrah.  I have a fix for my mysterious storm caching bug.  My duplicate_of changes can now land.
<deryck> gary_poster, would you have time for a review of this?  Since I think you chatted with lifeless about this duplicate_of caching issue when it was discovered.
<gary_poster> on calls, for afternoon :-/  can tomorrow
<deryck> ah, no worries.  Thanks anyway, gary.
<lifeless> moin
<lifeless> deryck: excellent
<deryck> lifeless, hey.  Glad you up.  Let me paste you a diff for now and see if you think I'm sane, or not. :-)
<deryck> lifeless, http://pastebin.ubuntu.com/473667/
<lifeless> that seems like a decent workaround if storm trunk doesn't have a fix
<lifeless> I'd *prefer* a new storm and no cruft, so perhaps you can add a XXX: and a bug reference there ?
<lifeless> jkakar: hi
<lifeless> jkakar: ^
<deryck> lifeless, sure.  I never could connect with anyone yesterday on storm, but the latest release on Launchpad is a few months old.
<deryck> So I can XXX until there's a fix in Storm.
<lifeless> yeah
<lifeless> I know it broken in the latest release as of 2 weeks ago
<lifeless> it might be fixed in trunk
<deryck> lifeless, I can report a bug if you didn't, but since you worked it out initially, if you have more to explain about it, feel free to report it. :-)
<jkakar> lifeless: Hi. :)
<deryck> oh
<deryck> unless it's fixed
<lifeless> deryck: ah rev 361 in trunk
<lifeless> jkakar: hiya. Has there been a storm release since rev 361 ?
<lifeless> deryck: http://bazaar.launchpad.net/~storm/storm/trunk/changes
 * deryck looks
<jkakar> deryck: Nope, I'm waiting for a branch to land (for you guys).
<jkakar> lifeless: ^^
<jkakar> lifeless: Then I'll prepare an 0.17 release.
<lifeless> ok, well we can use a snapshot
<lifeless> jkakar: whats the branch ?
<jkakar> lifeless: https://code.edge.launchpad.net/~jkakar/storm/sqlobject-is-empty/+merge/31565
<jkakar> lifeless: If you can take a peek and review that today, I can prepare 0.17 tomorrow.
<lifeless> jkakar: I'm with stub, it looks good
<deryck> interesting.  hi, jkakar, btw :-)
<jkakar> lifeless: Cool.  I'll prepare the release tomorrow.
<jkakar> deryck: Hi! :)
<deryck> lifeless, so should I hold on my changes here?  Or for now it's ok, until we get on latest storm?
<lifeless> deryck: latest storm is < 24 hours away
<james_w> hi lifeless, are you landing an updated version of testtools in to launchpad?
<lifeless> james_w: yes, if my merge is approved
<james_w> lifeless: great, I'll try and remember to make the Annotate change after that then
<lifeless> james_w: is my merge approved? Is that my cow?
<deryck> lifeless, yeah, but pqm closes tomorrow.  And I want a little time for QA with these changes.  Maybe I'm rushing too much, though.
<james_w> lifeless: you know about lp:lp-source-dependencies?
<lifeless> deryck: so, I'd be happy about you doing it via a versions.cfg change with a storm snapshot of trunk
<lifeless> deryck: because then your malone model code doesn't need to get all ugly at all ;)
<deryck> heh
<deryck> fair enough.
<lifeless> deryck: if you've done that before its very straight forward
<lifeless> james_w: lets pretend I don't
<deryck> lifeless, I haven't for storm.  But for lazr-js several times.  I assume it's similar.  Roll the egg, update versions.cfg, profit.
<lifeless> deryck: yeah
<lifeless> shove it in the download cache
<deryck> ok, branching now.
<lifeless> deryck: you can put the versions.cfg change in your branch for simplicity ;)
<lifeless> james_w: https://code.edge.launchpad.net/~lifeless/launchpad/registry/+merge/31830 is my new testtools referencing version
<james_w> lifeless: cool, thanks
<deryck> lifeless, so off tip, i.e. r365 of storm trunk?
<lifeless> deryck: looks like it should be sane to me
<deryck> cool, thanks!
<lifeless> de nada
<lifeless> I'm sorry I lost touch with where the fix was
<lifeless> or I could have said this on wed
<deryck> no worries.  Lot going on lately.
<lifeless> oh, thats steadystate for me :)
<deryck> The life of a TA :-)
<lifeless> ahha
<lifeless> no, *me*
<lifeless> this may mean I am a good fit for ta :P
<lifeless> jcsackett: so, I have https://code.edge.launchpad.net/~lifeless/launchpad/registry/+merge/31830 landing now
<sinzui> high lifeless, jcsackett and I started looking at the pathological cases of +participation
<sinzui> The same people keep getting listed, each has more than 150 teams
<lifeless> jcsackett: and if you look at that webpage I have the test I've written to show when the bug is fixed
<lifeless> sinzui: wheee
<lifeless> sinzui: so there may be rather different things for the /participation and +participation cases?
<lifeless> sinzui: thanks for your thoughts-on email, I think its a very good question to raise
<sinzui> +participation can die iterating over a large number of teams. The timeout can happen getting the icon, mailing list, or the path of the indirect team
<jcsackett> sinzui: and you said there's not a whole lot of people who are bringing this problem up, right?
<lifeless> sinzui: I'm just looking at one of the oops again now
<sinzui> pitti, sabdfl and mdz appear through this month. pitti every day
<lifeless> ok, I'm back up to speed
<lifeless> the /participation and +participation bugs are different but similar
<lifeless> from what I see they both are doing a lot of individual attribute lookups, with single backend sql queries happening as a result
<lifeless> the individual queries are fast
<sinzui> the mailing list query is death by a 1000 cuts
<lifeless> but the overhead of doing those queries is substantial, and when hundreds of queries show up, the page will be in trouble
<lifeless> sinzui: yes
<lifeless> so, jcsackett - I have not been working directly on +participation; I can offer thoughts about it, but if you're already dived-into it, you probably have more thoughts than me already :)
<lifeless> jcsackett: also, first time we've spoken - welcome to the team!
 * sinzui really wanted to know who was subscribed to lists.
<jcsackett> thanks, lifeless. glad to be on it.
<deryck> lifeless, can I get a rubber stamp please?  https://code.edge.launchpad.net/~deryck/launchpad/update-storm-r365/+merge/31881
<lifeless> deryck: r=lifeless
<deryck> lifeless, thanks!
<lifeless> deryck: however, do you want the bad news :)
<lifeless> deryck: its may conflict with my versions.cfg change
<deryck> ah, that's not so bad then :-)
<deryck> you had me scared :-)
<lifeless> :)
<deryck> I'm running through ec2 test now, so I'll merge trunk later tonight before lp-land'ing it.
<deryck> and resolve then
<lifeless> sinzui: ok, so do you / jcsackett have a plan, or do you want to talk about strategies to fix it?
<jcsackett> lifeless: i think sinzui and i are circling around a plan.
<lifeless> jcsackett: cool! I'm interested - I'm new to my role too :) - and trying to get a good feel for how things are fixed and improved
<jcsackett> lifeless: cool! yeah, i'm still getting a handle on things myself.
<sinzui> lifeless, we are exploring 1 batching for pathological cases, 2 make mailing cheap...possible get everything on one query, 3, consider the cost of building the path of indirect teams...we do extra work to avoid showing private teams
<lifeless> sinzui: 2 is what I would reach for first
<lifeless> sinzui: getting things down to a constant number of queries will help a great deal across the system
<flacoste> sinzui: are the mailman integration tests still off?
<sinzui> yes
* lifeless changed the topic of #launchpad-dev to: Launchpad Development Channel  | Week 3  of 10.08 | PQM will be closing 22:00 UTC Friday | firefighting: testfix mode | buildbot/pqm has been switched to watching the *lucid* builders | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<rockstar> We need to teach mup about testfix mode.
<lifeless> flacoste: I will land poolies missing patch today
<flacoste> lifeless: thanks
<flacoste> sinzui: this might be problematic for the Lucid upgrade
<mwhudson> morning
<flacoste> sinzui: your team should run the tests manually on Lucid and probably do a smoke test on staging (which is now on Lucid)
<sinzui> flacoste, we can enable them, but they never ran reliably. I ran them two months ago successfully, but bac or edwin got failures
<rockstar> flacoste, what are the ramifications of testing through ec2 now?
<flacoste> sinzui: i think running them once is fine, no need to enable them for all test runs
<flacoste> sinzui: if you get one succesful test run, it's likely to be fine
<flacoste> rockstar: i don't understand the questions, and how are you feeling btw?
<sinzui> flacoste, I will do that in a few hours then on two machines to verify
<flacoste> sinzui: it's not urgent, there are still many machiens to update, i've told mthaddon to hold on the mailman box until he gets confirmation from you that it's fine
<rockstar> flacoste, still feeling ill, but need to work to maintain my sanity.  Since Lucid failures can put us in testfix, how can I prevent my changes from doing so if the ec2 machine is on hardy?
<flacoste> rockstar: run "relevant" tests locally on lucid?
<flacoste> rockstar: that's no silver bullet, but i don't have any better suggestion until mars gets to update the ec2 image
<flacoste> rockstar: or someone else beats him to it
<rockstar> flacoste, ack.  I guess we'll just deal with the pain and embarassment of breaking the build when it happens then.  :)
<flacoste> :-)
<sinzui> jcsackett, look at https://edge.launchpad.net/~pitti/+participation
<thumper> jcsackett: morning
<thumper> jcsackett: welcome to the team
<jcsackett> thumper: good afternoon. :-) and thanks.
<sinzui> jcsackett, https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1678EC1214
<wgrant> Can someone please land https://code.edge.launchpad.net/~wgrant/launchpad/multi-arch-builders/+merge/31740?
<james_w> wgrant: yep
<wgrant> james_w: Thanks.
<james_w> wgrant: sbuild really takes the arch twice in its arguments?
<wgrant> james_w: sbuild-package is a wrapper. It does some stuff (and needs to know the architecture to do that), then passes the rest of the args through to sbuild.
<james_w> ah, ok
<wgrant> But it'll be gone once I land my system sbuild branch.
<james_w> wgrant: if you could cashttps://code.edge.launchpad.net/~james-w/launchpad/soyuz-factory-improvements/+merge/31890 and https://code.edge.launchpad.net/~james-w/launchpad/no-more-sampledata-2/+merge/31891 that would be greatt an eye over
<james_w> wgrant: that branch has detached in to ec2
<wgrant> james_w: Thanks. Looking at yoursnow.
<wgrant> james_w: https://code.edge.launchpad.net/~wgrant/launchpad/destroy-publishedpackage/+merge/31813 would also like to be landed, if you could.
<wgrant> james_w: In no-more-sampledata-2, you seem to prefer rSP over logging in as the right user.
<wgrant> I'm not sure if that's right.
<james_w> wgrant: in any particular place, or all over?
<wgrant> james_w: Setting parent_series is the most obvious so far.
<wgrant> You could also probably use BPR.override on line 201 of the diff.
<james_w> wgrant: I assumed that wasn't a normally allowed operation
<wgrant> james_w: It's exposed on DistroSeries:+admin.
<wgrant> So someone's allowed to do it.
<james_w> I'm happy to change if it's the right thing to do, I thought that in test setup it didn't really matter, as it's the results rather than the method that you care about
<wgrant> True.
<wgrant> What do all those .sync() calls do?
<wgrant> I wonder if they're irrelevant now that [SB]PPH isn't a view
<james_w> wgrant: that's possible
<james_w> I'd be tempted to just delete them and run the tests
<wgrant> james_w: I think that's probably a good idea.
<wgrant> james_w: Did you fire off that second ec2 instance?
<james_w> wgrant: second branch in ec2
<wgrant> Heh, thanks.
<james_w> wgrant: I'll give it a go
<james_w> must go and cook now though
<wgrant> There's still lots of code around that is redundant now that S[SB]PPH are gone.
#launchpad-dev 2010-08-06
<rockstar> wgrant, removing old code is the easiest way to get a branch landed.  :)
<cody-somerville> wgrant, S[SB]PPH is gone?!
<wgrant> cody-somerville: Yes, Julian removed them a few months ago.
<wgrant> makes everything muuuuch easier.
<wgrant> rockstar: True, but it's scattered everywhere.
<cody-somerville> hah. sweet.
<wgrant> And when I try to remove things, I inevitably end up getting fed up with the shocking Soyuz tests, so rewrite them instead...
<jkakar> lifeless: If you can get someone to bump the build priority of the new Storm packages you'll have your release earlier. :)
<rockstar> wgrant, yes, I know that dance.
<jkakar> I wonder what would happen if each team had to rotate every cycle.
<jkakar> All of a sudden Soyuz becomes Code, Code becomes Registry, Registry becomes Translations, etc.
<rockstar> jkakar, madness I tell you. Madness.
<jkakar> rockstar: Sure, but would it be *good* madness?
<rockstar> jkakar, depends what team you end up being on.
<rockstar> jkakar, also, ftr, the lines between the code team and the soyuz team have been blurred recently.
<wgrant> We managed to fob lots of our code off into collective maintenance at the start of the year.
<wgrant> I was pretty pleased.
<wgrant> Yes, exactly.
<jkakar> rockstar: That's great to hear.
<wgrant> jkakar: Not for Code it isn't :P
<jkakar> On the Landscape team we have no internal boundaries.  Everyone is expected to know everything (which of course isn't true, but no one can ever say 'Not my problem').
<jkakar> wgrant: :)
<wgrant> jkakar: How big is the Landscape codebase?
<jkakar> I sometimes wonder what would happen if that was the case in Launchpad.
<lifeless> in theory we have no/few internal boundaries
<wgrant> lifeless: Ha ha ha sure.
<lifeless> in practice we do
<lifeless> wgrant: jml and I, and I think francis, would like to see less partitioning
<wgrant> The team structure is defined to have subteams for each application!
<wgrant> That is surely creating boundaries.
<lifeless> wgrant: conways law, and yes.
<wgrant> lifeless: That's good.
<jkakar> wgrant: Landscape server has 235413 lines of Python, Javascript and ZPT.  Landscape client has 44351 lines of Python code.  A secret thing has 19197 lines of Python code.  A total of 298961 lines of code.
<jkakar> wgrant: Not that lines of code is really a measure of much.
<jkakar> Probably much smaller than Launchpad.
<lifeless> nah
<lifeless> sloccount of lib/lp and lib/canonical
<lifeless> python:      278120 (75.66%)
<lifeless> xml:          86102 (23.42%)
<lifeless> perl:          3002 (0.82%)
<lifeless> sh:             378 (0.10%)
<wgrant> 392735 Python, JavaScript and ZPT.
<lifeless> its both larger and smaller than it needs to be - larger on the source side, smaller on the tests side
<wgrant> Yep.
<jkakar> We have ~7900 tests in Landscape, ~1800 tests in Landscape client and about ~700 in secret thing.
<jkakar> In general we tend to have 25%-40% more test code than application code.
<lifeless> you really should rotate to the bzr team for 2 months
<lifeless> :)
<jkakar> That would be fun. :)
<jkakar> I guess you have a lot more test code than that?
<lifeless> 21K tests run in trunk, or there abouts
<lifeless> there is a multiplicative effect there due to test scenarios per-backend, but still
<jkakar> lifeless: By the way, I've been looking at testscenarios for a project I'm hacking on.  It's really nice.  Thanks. :)
<lifeless> jkakar: my pleasure
<lifeless> jkakar: I've another one up my sleeve.
<lifeless> Just need a weekend to realise it
<jkakar> Nice.
<lifeless> jkakar: you might like this too - https://code.edge.launchpad.net/~lifeless/launchpad/registry/+merge/31830
<jkakar> lifeless: Very nice.  I've been looking at matchers more closely too.  I'm excited, they solve lots of annoying problems.
<jkakar> lifeless: Need to integrate them in Landscape still.
<lifeless> jkakar: \o/
<lifeless> jkakar: assertThat in testtools is:
<lifeless>  - small
<lifeless>  - BSD/ApacheV2
<lifeless> so just grab it and shove it in your base clsas
<jkakar> That's my plan. :)
<jkakar> In fact, I've been doing that to use it with Twisted.
<lifeless> you could put it in twisted ;)
<cody-somerville> Has anyone seen Review Board before?
<lifeless> yes
<cody-somerville> It seems really slick. Is that the direction launchpad reviews are going in?
<lifeless> its a bit of an ambiguous question
<lifeless> yes, clearly everyone wants to make lp reviews better
<lifeless> is it going to be a clone of rb? I doubt it.
<jkakar> lifeless: https://edge.launchpad.net/storm/trunk/0.17
<cody-somerville> I really like how it lets you embed snippets of the patch in your comment and have it highlighted and lets you comment on the specific comments against a snippet in a comment
<lifeless> cody-somerville: feel free to file wishlist bugs for specific things you'd like to see
<lifeless> jkakar: hey, can we have a voice chat?
<jkakar> lifeless: Sure thing.  Skype?
<lifeless> I want to talk through mapping ORM failure modes
<lifeless> yes
<jkakar> lifeless: I'm 'jkakar' on Skype.
<lifeless> you need to click 'allow' not 'sodoff'
<lifeless> :P
<poolie> hi lifeless, i'm planning to do the next tranche of flags today
<poolie> unless there are any priority interrupts
<lifeless> poolie: cool
<poolie> lifeless: brief call?
<lifeless> otp right now
<lifeless> after that sure
<lifeless> poolie: ok, off the phone, ping me when you want to chat
<lifeless> sinzui: bug 593054 - how would you feel if I land a patch dropping the memcache use from it ?
<_mup_> Bug #593054: Need to flush caches with modified object <Launchpad Foundations:Triaged> <Launchpad Registry:Triaged> <https://launchpad.net/bugs/593054>
<lifeless> poolie: I might go cook some lunch and so on, but I dont' want to block you
<poolie> no i'm good now
<lifeless> ok. skype ?
<thumper> ââ¹
<thumper> mwhudson: the merging of the codehosting areas may have landed everywhere except lp.dev
<lifeless> poolie: your branch [with my tweaks] fails some tests. forwarding to you
<mwhudson> thumper: ?
<thumper> mwhudson: using make run_codehosting we get branches in two locations:
<thumper>  /var/tmp/bazaar.launchpad.dev/push-branches/
<thumper> and
<thumper>  /var/tmp/bazaar.launchpad.dev/mirrors/
<thumper> or at least pushing puts it in mirrors
<thumper> and the makefile makes branches in push_branches
<thumper> I'm trying to work out why the scanner is looking in the wrong place
<mwhudson> oh oops
<mwhudson> i sort of vaguely knew about make-dummy-hosted-branches being wrong
<thumper> it could just be the make-dummy... thing that is wrong
<thumper> and the reason that the scanner is broke
<thumper> is that I've not installed the local launchpad recently
<thumper> mwhudson: why is the scanner looking in the wrong place?
<thumper> I can't figure it out
<thumper> it uses lp-internal
<thumper> gah
 * thumper smacks head against the table
 * thumper still can't see it
 * thumper leaves for a bit
<mwhudson> thumper: lp-internal maps to http, so that should be served from ../mirrors
<thumper> mwhudson: which has me wondering why the scanner can't open it
<mwhudson> thumper: what is the scanner saying?
<thumper> TransportError: Transport error: Server refuses to fulfill the request (403 Forbidden) for http://bazaar-internal.launchpad.dev/00/00/00/50/.bzr/branch-format
<mwhudson> sometimes /var/tmp/bazaar.launchpad.net/mirrors isn't world readable
<mwhudson> thumper: ah, bet this is the case
<mwhudson> if you can figure out why this happens...
<thumper> ah
<thumper> (*&%^$#@!#$%^&
<thumper> mwhudson: that's it
<thumper> mwhudson: I'll fix the config stuff in a different branchj
<poolie> thanks lifeless
<lifeless> I need a teddy-bear who knows APIs well
<lifeless> anyone around ?
<lifeless> specifically, I have a property which is exported and I'd like to know how to make it call a function rather than just accessing a property
<wgrant> lifeless: What do you mean?
<lifeless> wgrant: so, this is 'allmembers'
<lifeless> 'allmembers' is exposed as /participation on people/teams
<lifeless> for many uses, what the query retrieves today is fine, I don't want to make it greedy automatically.
<wgrant>  /participants, you mean?
<lifeless> but for this particular case, because its being rendered as json
<lifeless> blah yes
<wgrant> You need to export a method which does it instead.
<lifeless> I want to expand out all the things that it gets so the json rendering etc
<wgrant> Then callers have to call the method.
<lifeless> wgrant: right, but 1.0 api freeze.
<wgrant> You can't, unfortunately, just make the attribute lazy.
<lifeless> wgrant: other way around
<lifeless> anyhow, I think I have it
<lifeless> stop exporting allmembers
<lifeless> export allmembers_full or someting 'as' participants
<lifeless> have allmembers and allmembers_full both call a common core function with arguments expressing how greedy they want the object evaluation to be.
<wgrant> If you export an attribute at all, it will be serialised along with the rest of the representation.
<lifeless> only fields
<wgrant> Oh, right.
<lifeless> this will be a CollectionField
<lifeless> so not included
<wgrant> Your approach sounds sane.
<lifeless> cool, thanks
<poolie> i'm getting this error trying to start postgres in a chroot dedicated to launchpad
<poolie> [2010-08-06 07:26:08 UTC] FATAL:  could not create shared memory segment: Invalid argument
<poolie> [2010-08-06 07:26:08 UTC] DETAIL:  Failed system call was shmget(key=5432001, size=39395328, 03600).
<poolie> does this ring any bells?
<lifeless> uhm
<lifeless> very very vaguely
<lifeless> do you have two chroots like that at the same time? likely to fail I think
 * lifeless uses vms because of this ;)
<StevenK>     warnings.warn(UnreasonableRemoveSecurityProxyWarning(obj), stacklevel=2)
<StevenK> UnreasonableRemoveSecurityProxyWarning: Called removeSecurityProxy(<PackageUpload at 0xce434d0>) without a check if this is reasonable.
 * StevenK pouts
<spm> lifeless: vms as in Virtual MachineS? not VAX-VMS?!??!
<poolie> you're showing your age uncle steve
<lifeless> spm: have I told you about a classmates sort algorithm
<poolie> lifeless: i'm pretty sure nothing else has it bound
<lifeless> spm: on the otago uni vax
 * spm was a vms sysadmin for 5 years!
<lifeless> spm: he wrote a sort, which he called 'randomsort'.
<spm> or 6. I forget...
<lifeless> spm: what do you think the supervisor process did to it ....
<poolie> i'ts possible something else is using a lot of shm
<spm> promoted to always runnable and DOS'd itself!
<lifeless> \o/
<lifeless> yes, he got slapped :)
<spm> vms shell script killer 101: 10: i=i+1;  20: goto 10. <== server dies.
<spm> istr that little gem was fixed in the early 90's, with whatever version came out then. but don't recall details.
<lifeless> this was 93 :P
<spm> 94 is still early! ;-)
<spm> I think DEC had the optimistic viewpoint of their mostly boriing corporate customers - if you're dumb enough to that to yourself; sucks to be you.
<adeuring> good morning
<spm> heya adeuring!
<StevenK> Right, now tests I didn't write fail due to that warning. :-(
<wgrant> StevenK: The warning shouldn't be making them fail...
<spm> StevenK: if any consolation, staging is now kaput; and one of the edge servers has also died; I'd suggest in a near identical way to staging. Personally I think it's all your fault; but have no proof... yet.
<StevenK> Error in test lp.archiveuploader.tests.test_ppauploadprocessor.TestPPAUploadProcessorQuotaChecks.testPPASizeQuotaSourceWarning
<StevenK> Traceback (most recent call last):
<StevenK> ...
<StevenK> UnreasonableRemoveSecurityProxyWarning: Called removeSecurityProxy(<PackageUpload at 0xce434d0>) without a check if this is reasonable.
<wgrant> spm: Is that due to the vostok changes?
<spm> wgrant: bingo
<StevenK> Vostok is so not my fault
<lifeless> is too
<spm> sure it is. I said so. QED. so did Rob, QED x 2.
<StevenK>  /ragequit
<spm> rofl
<spm> is there a fix for that in the pipeline? or do I need to hand revert staging back to a level of workingness?
<wgrant> spm: mwhudson has a fix. It was apparently in EC2 at 8am.
<wgrant> But hasn't shown up in devel..
<spm> woot
<wgrant> Oh, wait.
<wgrant> I think it did land.
<spm> i recall him talking about something, but never did get the details
<wgrant> devel r11307
<spm> 304 is what hit edge
<wgrant> That's current stable.
<wgrant> It's merging oddly.
<wgrant> Two stable->db-devel merges, less than an hour apart.
<wgrant> I don't see how that could happen.
<spm> <frown>
<poolie> lifeless/others, when you develop lp in a vm, do you simply keep your source tree within the vm?
<lifeless> yes
<lifeless> I use ssh -A
<lifeless> and virsh start/shutdown
<poolie> and run your editor etc in the vm too?
<lifeless> yes
<lifeless> of course, as I use vim, not gvim, normally, this is no change for me ;)
<lifeless> mwhudson: /var/tmp/vostok-archive
<lifeless> mwhudson: why is 'make' making that?
<wgrant> Oh, are there lucid and non-lucid builders both blessing devel and db-devel at the moment?
<lifeless> that seems likely
<lifeless> also fail
<lifeless> can you tell mars ?
<mthaddon> wgrant: there are lucid and non lucid builders that are both tracking devel and db-devel, but in terms of whether we're in "testfix" mode, we only care about the lucid ones
<wgrant> mthaddon: But both seem to be able to bless devel for merging into db-devel.
<mthaddon> hmm, interesting - I think that's probably true, yeah
<bigjools> wgrant: did you get my PM about that failing test in your branch?
<wgrant> bigjools: I did. I fixed it, and it's landed.
<wgrant> Thanks.
<bigjools> greatr
<lifeless> stub: is there a wiki page about the workflow for doing db patches?
<stub> Yes. schemachanges or something. Its linked off the processes area I think.
<bigjools> wgrant: are you still working on https://code.launchpad.net/~wgrant/launchpad/rescue-aborted-and-robbed-builders/+merge/22289
<deryck> Morning, all.
<jml> morning
<lifeless> hai and hai
 * lifeless hopes mars pops in soon
* bigjools changed the topic of #launchpad-dev to: Launchpad Development Channel  | Week 3  of 10.08,  Release Manager: bigjools | PQM will be closing 22:00 UTC TODAY | firefighting: testfix mode | buildbot/pqm has been switched to watching the *lucid* builders | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pas
* bigjools changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 3 of 10.08, Release Manager: bigjools | PQM will be closing 22:00 UTC TODAY | firefighting: - | buildbot/pqm has been switched to watching the *lucid* builders | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<wgrant> bigjools: Didn't I throw a comment on that this morning?
<wgrant> It bitrotted completely post-Wellington, so someone else should take it.
<bigjools> ok
<bigjools> can you mark it abandoned?
<bigjools> wgrant: also looking at your ddebs for PPAs branch
<bigjools> does this need a buildd rollout?
<wgrant> bigjools: The buildd change was rolled out months ago.
<bigjools> ok
<wgrant> Can you reject my MP? I can't do that.
<bigjools> done
<wgrant> Thanks.
<bigjools> I need to address that problem soon, it's a royal PITA
<wgrant> Yes.
<bigjools> wgrant: I don't supposed you'd be able to create some upload scenarios to QA your dependency wait changes?
<wgrant> bigjools: Like, say, lamont's slony package?
<wgrant> That's one that should already be there.
<wgrant> But I could create another if you want.
<bigjools> I need it on dogfood
<bigjools> so I can run buildd-retry-depwait.py
<wgrant> bigjools: I guess you'll need to set all the existing depwait builds to something else, or you're going to have a rather flooded build farm...
<bigjools> wgrant: that's already sorted, I ran it a couple of days ago and then deleted the queue
 * bigjools taps side of nose sagely
<wgrant> But for easy setup, create a PPA with normal dependencies and copy slony1-psql8.3 from https://dogfood.launchpad.net/~lamont/+archive/psql8.3/+packages
<bigjools> ok
<wgrant> It will depwait, since it needs debhelper 7, and that's only in hardy-backports.
<wgrant> It's being retried every time the script runs at the moment.
<wgrant> That's a lot of manual builders.
<wgrant> Is glib still screwed?
<lifeless> maxb: gnight - would love it if you could drop me a mail saying where I could help with the sheperd project
<maxb> lifeless: tab completion fail?
<lifeless> maxb: bah, yes
<lifeless> mars: ^
<james_w> bigjools: am I ok to land https://code.edge.launchpad.net/~james-w/launchpad/stop-publishing-nagging/+merge/31851 ?
<jcsackett> anyone get an error on "Getting distribution for testtools==0.9.6dev91" when running "make build" ?
<jelmer_> jcsackett: you might have to run ./utilities/update-sourcecode
<jcsackett> jelmer_, that updated some things but the error remains. :-/
<jcsackett> thanks for pointing that out though, didn't realize that script was there.
<jelmer_> jcsackett: Do you have a testtools_0.9.6dev91.tar.gz file in download-cache/dist ?
<jcsackett> no, jelmer_,  it's 0.9.2; does that indicate update-sourcecode didn't run properly?
<jelmer_> jcsackett: I'm not sure, I think it might perhaps just update the bzr checkouts in sourcedeps/.
<jelmer_> jcsackett: You might want to try running "bzr up" in download-cache
<jelmer_> jcsackett: rocketfuel-get (if you use the standard lp directory layout in ~/launchpad/{lp-branches,lp-sourcedeps}) will also do all these things for you.
<jcsackett> jelmer_: ah, okay. i'll try that. thanks!
<jcsackett> jelmer_: looks like that did it (it's at least past where it was erroring before). thanks, again.
<bigjools> james_w: yes, PQM is not closed yet
<james_w> bigjools: I was more asking from a soyuz standpoint
<bigjools> ah sorry
<james_w> jml asked me to double-check with you
<james_w> also, when it's not release crunch I would like to discuss Soyuz and the factory with you
<bigjools> yes PackageUpload is only ever written to in zopeless scripts
<bigjools> so it's fine to remove its proxy
<bigjools> james_w: that would be in a week's time then :)
<jtv> henninge: I'm fazed.
<jtv> henninge: Scenario one:
 * henninge has 3 minutes to go ....
<jtv>         old_entry = queue.addOrUpdateEntry(
<jtv>             template.path, '# Content here', False, template.owner,
<jtv>             productseries=template.productseries)
<jtv>         new_entry = queue.addOrUpdateEntry(
<jtv>             template.path, '# Content here', False, template.owner,
<jtv>             productseries=template.productseries, potemplate=template)
<jtv> Two entries.  No worries.
<jtv> Scenario two:
<jtv>         old_entry = queue.addOrUpdateEntry(
<jtv>             pofile.path, '# Content here', False, template.owner,
<jtv>             productseries=template.productseries)
<jtv>         new_entry = queue.addOrUpdateEntry(
<jtv>             pofile.path, '# Content here', False, template.owner,
<jtv>             productseries=template.productseries, pofile=pofile)
<jtv> Boom!
<jtv> Crash!
<jtv> Violates unique constraint.
<jtv> The only thing I can think of is that pofile is not part of the constraint...
<henninge> jtv: pofile is not taken into account.
<henninge> jtv: that is the reason (I figures) why that wordpress-ckb.po did not get approved yesterday.
<jtv> Well this particular problem happens while creating the entry.
<henninge> jtv: When I went to the approval form, I still had to pick the template - although the entry already said "will be im ported into ..."
<jtv> AFAIK potemplate is normally null for translation uploads.
<henninge> jtv: right
<jtv> Anyway, the constraint indeed does not care about pofile.
<jtv> So that rules it out as one of the two possible causes of the bug I'm working on now, which is a bit of a relief.
<henninge> jtv: right, that's why the two entries are identical to it
<henninge> jtv: but your question is answered, right?
<jtv> henninge: yes, I answered it by explaining the problem.  Thanks.  :)
<jtv> danilos: https://code.edge.launchpad.net/~jtv/launchpad/bug-613821/+merge/31957 âI can't set the review type to "code" right now.
<jtv> danilos: I'd be much obliged if you move the kanban card about as wellâ¦
<danilos> jtv, I will, thanks
<jtv> danilos: I'll go do non-work stuff now, but will check back from time to time.  If/when you feel it's ready, please land it for meâ¦  I'm also running all Translations tests on it so we have reasonable certainty that it won't fail in EC2.
<danilos> jtv, did addOrUpdateEntry signature change? (or is it just whitespaces?)
<jtv> danilos: whitespace.
<danilos> jtv, cool
<jtv> I promise.
 * danilos rechecks
<danilos> jtv, does _getMatchingEntry return only an already approved entry? (i.e. it will not return "itself")
<jtv> danilos: it's only called when looking for the more specific match that the approver's pofile/potemplate guess will produce.
<jtv> So it won't produce the entry that you're already looking at: you'd be looking for an entry with the same pofile/potemplate, and in that case the caller shortcuts.
<danilos> jtv, it'd be nice to return an explicit False in _attemptToApprove when entry.status != NEEDS_REVIEW
<jtv> danilos: whoops, sorry :)
<danilos> jtv, also, your comment about the 'checking at the very end if it was already approved' confused me a bit :)
<jml> deryck, ready when you are.
<danilos> jtv, old code was not even considering that problem, it was checking if approval succeeded instead
<deryck> jml, ok, 3-4 minutes still 'til I'm ready.  Mumble?
<danilos> jtv, basically what you do with a check on the returned value from _attemptToApprove
<jml> deryck, sure.
<danilos> jtv, anyway, looks good other than that minor point above, r=danilo
<jtv> danilos: thanks.  I meant the part where it checked at the end whether the entry's status was APPROVED.  Not too sane.  :-)
<jtv> danilos: could you land it for me?
<danilos> jtv, right, but that's your "if success" check, roughly the same thing
<jtv> danilos: not entirely, which shows that this needed cleaning up.  Basically, in the old code, if the status is APPROVED then how did you get to that point?  By wasting time on an entry that was already approved.
<danilos> jtv, right, I am saying that you are adding a totally unrelated fix and your comment about how it's just moving a check to a different place confused me :)
<danilos> jtv, it's not moving a check, it's a new check that didn't exist before, and you still do a similar check for exactly the same reasons old code did :)
<danilos> jtv, anyway, can you please set the commit message to what you feel is appropriate? :)
<jcsackett> hi, sinzui.
<sinzui> hj jcsackett
<jtv> danilos: there's two different checks here.  One is "does this entry still need review?" and the other is "did I make any progress?"  I moved the former up to before the attempt to approve the entry.
<jtv> danilos: I can try to edit the commit messageâ¦ the UI wouldn't let me earlier, probably because I'm working through my phone.
<danilos> jtv, where was that "former" check before?
<danilos> jtv, ah, ok, I'll take care of that; have you fixed the "return False" and pushed that?
<sinzui> bigjools, ping. do you have time to talk about bug 607879 with jcsackett and myself
<jtv> danilos: fixed yes, just checking that it is indeed pushed.
<_mup_> Bug #607879: https://bugs.edge.launchpad.net/~person/+participation timeouts <timeout> <Launchpad Registry:In Progress by jcsackett> <https://launchpad.net/bugs/607879>
<jtv> danilos: the old check was:
<jtv>            if entry.status != RosettaImportStatus.APPROVED:
<jtv>                 there_are_entries_approved = True
<bigjools> sinzui: Sure.  I am feeling feverish, but I'd probably make the same sense as if I wasn't.
<sinzui> bigjools: jcsackett and I may ask for an RC for this oops. It is the leading oops in the registry domain.
<danilos> jtv, hum, I read that as the same check you did, except that I misread the '!=' part
<danilos> jtv, it seems you are changing the semantics of the method, now it returns
<jtv> danilos: never mind, it's all horrible
<danilos> True when there's something approved, and it used to return True when there was something it didn't approve
<sinzui> bigjools, We need to stormify two methods, and one will require updating the callsites to deal with the changed output
<jtv> danilos: only in the case where, while you were approving, an entry changed to some state other than APPROVED through external forces.
<sinzui> bigjools, So I think I have forewarned you about our goals for today
<jtv> (From NEEDS_REVIEW)
<danilos> jtv, I seem to be reading this wrong
<danilos> jtv, so, if an entry stayed NEEDS_REVIEW (i.e. we didn't find what to approve), we used to set there_are_entries_approved to True (yes, the var name is wrong)
<bigjools> sinzui: anything that fixes an oops is likely to get an RC from me :)
<jtv> danilos: well maybe I am, because it is messy.  But old situation: go through all the motions of approving, then check that the entry isn't approved already, and if not, set its status as Approved.
<jcsackett> bigjools: fantastic. :-)
<sinzui> bigjools, thanks.
<danilos> jtv, oh, not really, you are right
<jtv> danilos: see?  The old code was carefully designed to confuse and disorient intruders.
<danilos> jtv, anyway, it is horrible code
<jtv> Now that the Cold War is over, we can ease away from that.  :)
<danilos> jtv, heh, yes, a nice improvement there
<bigjools> sinzui, jcsackett: you're welcome.  Let me know how you get on.  I may be taking a few hours off and coming back later, I feel like death.
<danilos> jtv, anyway, thanks for the fix, I'll get it through ec2 and land
<danilos> bigjools, don't go!
<jtv> danilos: thanks.  FWIW it got through at least all the Translations pagetests and doctests and all the windmill tests I observed, as well as lp.translations.tests
<bigjools> danilos: can I help? :)
<jtv> bigjools: wait!  What _does_ death feel like?  We're all not quite dying to know.
<danilos> bigjools, no, no, I am good, I am just sad to see you go, get drunk and all those things
 * bigjools raises scythe and points at jtv
 * jtv ducks
<bigjools> danilos: I wouldn't be able to drink if I wanted to, my throat is on fire
<danilos> bigjools, oh well, get some rest man, I am sure you'll be back to check on PQM has-closed midnightish :)
<bigjools> you'd think :)
<mars> wgrant, still around?  You meantioned something about both builders blessing devel and db-devel?
<bigjools> mars: apparently that was the case earlier, I think mthaddon is aware?
<mthaddon> bigjools: not really - I haven't looked into it at all
<bigjools> oh ok, I thought wgrant had told you
<jml> jelmer_, what should I do?
<jml> jelmer_, trigger a rebuild or actually investigate the source of failure?
 * jml suspects an erratic test.
<jelmer_> jml: Perhaps trigger a rebuild first?
<jelmer_> could the mocking about with the buildbots and their recent restart perhaps be related?
<jml> 4 hrs, 4 mins, 56 secs at 20:52:00
<jml> jelmer_, probably not.
<jml> I suspect the root cause of the problem is TacTestSetup being terrible.
<jml> which is actually a mask for a deeper problem with Twisted
 * jml frowns
<jelmer_> oh, db-devel just broke too, but with different errors
<jml> yeah, but danilos warned us about that.
<danilos> jelmer_, jml: the fix for that is already in db-devel so I am just forcing a build
<danilos> (the same will happen on lucid-db-lp which will actually get us into testfix mode again)
<jml> danilos, sweet.
<danilos> (this should at least minimize the testfix mode for everyone, and again, I apologize for your potentially failing db-devel ec2 test runs)
<sinzui> Anyone available for a quick vote on milestone caching? The choices are remove it all, or leave it for anonymous only
<danilos> sinzui, my vote would be that whoever goes and does it decides what to do :) (i.e. weights the pros and cons on both: there's no perfect answer, so voting won't really help much)
<sinzui> I prefer keeping anon cache. they cannot change something to see it change. The tests need minor updates to record the cache type.
<mars> wgrant, I think we figured out the db-devel landings: I /think/ PQM stopped processing submissions for about 5 hours, so both db-devel landings ended up in the queue, landing right beside each other.
<jelmer_> sinzui: I think caching for anonymous users makes sense, as a developer it's confused me a few times
<sinzui> thanks jelmer
<jml> grr... how do I use pip to install a package from a tarball.
<benji> jml: you can probably tell it the directory with the tarball in it is your index
<jml> benji, thanks.
<benji> jml: oh, wait, it's easier than that: "pip install path/to/mypackage.tgz"
<benji> from http://guide.python-distribute.org/usage.html#installing-from-a-tarball
<jml> benji, but after my frustrated outburst I just untarred it and installed it with good ol' python setup.py install.
<benji> heh
<jml> benji, thanks. good to know both the doc & the command for the future
 * rockstar lunches
<jkakar> deryck: btw, I release Storm 0.17 yesterday.
<jkakar> deryck: Do you need anything else before you can upgrade the version of Storm Launchpad is using?
<jkakar> deryck: The release is here: https://edge.launchpad.net/storm/trunk/0.17
<deryck> jkakar, nope, that's perfect.  Getting my branch blessed now to update us.
<jkakar> deryck: Awesome!
<sinzui> Does anyone know how to make a view render() with ANONYMOUS in a unit test? passing principal=None does what I expect to the request, but render fails wanting a logged in user
<salgado> sinzui, principal=UnauthenticatedPrincipal, maybe?
<sinzui> salgado, None cause this to be set:
<sinzui> request.setPrincipal(
<sinzui>             getUtility(IPlacelessAuthUtility).unauthenticatedPrincipal())
<sinzui> salgado,  I think the request is correct, but render() ...
<sinzui> oh, maybe I need to login as ANONYMOUS as well as call render with principal=None
<salgado> isn't your view expecting a user to be logged in?  iow, is it not a launchpad.AnyPerson view?
<sinzui> It is a milestone view and I am trying to update the test to show memcache is only enabled for anonymous uers
<sinzui> woot!
<sinzui> salgado, I had to login(ANONYMOUS), I was actually logged in as another user from the test setup. I had written the create_initialize_view to have none as the principal...and assumed the latter was the issue
<lifeless> mars: hi?
<kiko> lifeless, would you care to look at some bzr bugs loic filed and see what you think?
<lifeless> url me up
<kiko> https://launchpad.net/bugs/593560
<_mup_> Bug #593560: Slow performance for many operations on the gcc code import branches <udd> <Bazaar:Confirmed> <https://launchpad.net/bugs/593560>
<kiko> https://launchpad.net/bugs/605067
<_mup_> Bug #605067: want option to allow uncommit but disallow changing mainline <amd64> <apport-bug> <maverick> <Bazaar:Confirmed> <bzr (Ubuntu):Confirmed> <https://launchpad.net/bugs/605067>
<lifeless> so the second I'd already seen, I agree it would be good
<lifeless> probably fairly easy too
<lifeless> the first bug I've seen too
<lifeless> its a bit of a problem bug because its very very wide
<kiko> yes, it's annoying
<lifeless> we've talked just about the new-tree creation time
<lifeless> which is being addressed - john & martin were doing a patch @ the rally to disable accelerator trees (which in this case are a pessimisation), and john has been working on performance on the gcc tree ever since
<lifeless> it might be good to file a series of targeted bugs
<kiko> lifeless, oh, that's really good to hear -- I think he and loÃ¯c didn't sync up then
<lifeless> and andrew has just landed a patch that also helps as well
<lifeless> by being leaner on the ie objects
<kiko> yeah, the one that mark wrote to us about :)
<lifeless> right
<lifeless> jam's work loads the dirstate faster and reduces memory load from the chk pages
<lifeless> its probably going to be 2.3 only
<kiko> lifeless, do you know when that's due?
<lifeless> 2.2 was just released
<lifeless> so 2.3 betas will be coming out soon, and 2.3 itself 6 months
<kiko> hmm
<kiko> "that's a long time to wait" :-)
<lifeless> well
<kiko> lifeless, I had one user, ams_cs, who had a commit take 30 minutes the other day, and a few take a couple of minutes
<lifeless> you can track betas
<kiko> I was surprised -- do you think there's anything pathological he's doing?
<kiko> yeah, I should tell them to
<lifeless> uhm, its possible that that 30 minute was a complete repack
<kiko> yes, that's what he said he thought it was
<lifeless> which happens when you click over to a new power-of-10 commits
<lifeless> so, if he hit the 1000000
<kiko> aha
#launchpad-dev 2010-08-07
<mars> lifeless, hey, still around?
<lifeless> yeah
<mars> lifeless, I have a minute or two.  What's up?
<mars> lifeless, we had a bunch of discussion today about the qa-shepherd thing - cut the scope down again, it should be ready sooner than what we expected before
<mars> lifeless, now that we have done that, I can send you that email update you asked for
<lifeless> mars: I was offering to dive in and help
<lifeless> mars: rather than a status update
<mars> lifeless, well, now we have the scope down to where the three of us have to worry about stepping on each-other's toes :)
<lifeless> nice
<mars> lifeless, thank you for the offer, I'll keep it in mind.  I think we'll be fine for the first few days of next week.
<lifeless> ok cool
<lifeless> (I don't do 'status update requests' unless I -really- can't help it - just for clarity :)
<mtaylor> lifeless: shouldn't you be on saturday?
<lifeless> yes
<mars> lifeless, makes sense :D
<mars> mtaylor, it's not Saturday here, and he asked first :)
<mtaylor> weird PPA build issue- only shows up on i386 lucid builder:
<mtaylor> dpkg-deb: parse error, in file 'debian/drizzle/DEBIAN/control' near line 6 package 'drizzle':
<mtaylor>  `Depends' field, reference to `drizzle-server': version contains ` '
<mtaylor> mars: hehe. his fault then
<lifeless> mars: so generally the fastest way to make something go faster is to help :- which is why I offered ;>
<mtaylor> http://launchpadlibrarian.net/53201197/buildlog_ubuntu-lucid-i386.drizzle_2010.08.1683-1ubuntu1~lucid0_FAILEDTOBUILD.txt.gz  in case anyone wants to look
<lifeless> mars: still around?
<mars> lifeless, yep, thinking
<lifeless> mars: so if there isn't anything sensible I can hop in and do; don't bother mailing me ;)
<mars> lifeless, sound fair to me.
<mars> sounds even
<mars> need to run again
<mars> lifeless, I'll ping you back with specifics if I get a moment back at the keys
<mars> lifeless, at the keys, if you are still there, skype?
<lifeless> mars: hi
<mars> lifeless, yep?
<lifeless> I amhere
<mars> skype?
<mars> it's faster, trust me
<mars> lifeless, https://code.edge.launchpad.net/~ursinha/qa-tagger/setuptools-conversion
<mars> lifeless, https://code.edge.launchpad.net/~launchpad-qa/qa-tagger/devel
<mars> lifeless, http://bazaar.launchpad.net/~launchpad-qa/qa-shepherd/devel/annotate/head:/shepherd/testing.py
<mars> https://bazaar.launchpad.net/~ursinha/lp-qa-tools/lplib-fakes/files
<lifeless> make check working for me now :)
<mars> lifeless, btw, feel free to move the cards on the board if you get a chance to hack on something.  That way we can pick or pass the work on Monday morning.
<mars> "we" being those of us on the opposite side of the planet from you :)
<lifeless> :)
<cody-somerville> How can you tell when its more efficient to do separate queries and when its more efficient to just do a single query with joins
<cody-somerville> ?
<lifeless> the science of measurement
<lifeless> also some context will help
<lifeless> there are places where its always better to do it one way or another.
<lifeless> e.g. per-object queries in an iterator for an api or web page -> always want one query ('build the table of data to show')
<lifeless> this is a scaling question - ask yourself what happens when you have BigN (e.g. 400 people on one page)
<cody-somerville> I was just wondering if there was a quick and easy way to tell... like look at the output of EXPLAIN and look for this or that.
<wgrant> When's it going to be better to do multiple queris?
<cody-somerville> re: buildd-master failure, 2010-08-07 23:26:24+0100 [-] exceptions.AttributeError: 'NoneType' object has no attribute 'email'
<lifeless> cody-somerville: where do you get access to that ?
<lifeless> escalating to phone.
<wgrant> cody-somerville: Thanks.
<wgrant> cody-somerville: Is there something before that indicating which build it's trying to send a failure message about?
<lifeless> wgrant: different domain lookups that aren't related to each other
<cody-somerville> 2010-08-07 23:26:24+0100 [-] Starting templates build trunk-3771906 for lp:docky.
<cody-somerville> 2010-08-07 23:26:24+0100 [-] startBuild(Lucid, daily, gnome-terminator (on http://doubah.ppa:8221/))
<cody-somerville> 2010-08-07 23:26:24+0100 [-] Scanning failed with: 'NoneType' object has no attribute 'email'
<cody-somerville> lifeless, devpad
<wgrant> Aha.
<lifeless> cody-somerville: where - /srv/ somewher e?
 * wgrant prepares the SQL.
<cody-somerville> lifeless, /srv/launchpad.net-logs/soyuz/cesium/buildd-manager.log in this case, yea
<lifeless> cody-somerville: thanks
<lifeless> elmo will be around shortly
<elmo>  
<cody-somerville> wgrant, would having a launchpad admin set a preferred e-mail address for the requester work?
<wgrant> cody-somerville: It might, but we could also just suspend the build to preserve the evidence.
<lifeless> suspend it please
<lifeless> we need to fix root causes
<cody-somerville> I've ran into this issue myself with scripts that use the launchpadlib
<cody-somerville> I had assumed that every user had preferred email address I could grab.
<wgrant> elmo: SELECT * FROM buildqueue, job, branchjob WHERE buildqueue.id = 3771906 AND job.id = buildqueue.job AND branchjob.job = job.id;
<cody-somerville> Part of the issue is that you have to get the actual e-mail address from an attribute on the user that doesn't exist if there is no preferred e-mail address available
<wgrant> (I really don't know why a translations job dispatch would be trying to acesss an email address)
<cody-somerville> wgrant, its a recipe build thats causing it to die
<cody-somerville> wgrant, line 66 lib/lp/code/model/recipebuilder.py
<wgrant> Oh, recipe build logging is fucked.
<cody-somerville> lifeless, are you going to own the incident report?
<lifeless> cody-somerville: thanks for volunteering :)
<wgrant> elmo: You should have a cancel button on https://code.edge.launchpad.net/~gnome-terminator/+recipe/daily/+build/515
<cody-somerville> lol, alright. :P
<elmo> man, seriously
<elmo> this is Ng's fault?
<lifeless> its a bug in translations somewhere
<wgrant> No, it's not.
<wgrant> The logs are just misleading. It's actually a recipe build.
<elmo> wgrant: I do?
<elmo> wgrant: sorry
<elmo> wgrant: I do.  hit it?
<lifeless> wgrant: oh, grah
<wgrant> elmo: Please.
<cody-somerville> wgrant, the recipe build belongs to Ng
<lifeless> anyhow,  bug, not user error, right ?
<wgrant> lifeless: The first two lines of that log that cody-somerville pasted are equivalent, but for different job types. Yay for consistency.
<wgrant> cody-somerville: It is, yes.
<wgrant> lifeless: Right.
<elmo> wgrant: done
<cody-somerville> wgrant, sorry, misread you
<wgrant> We're 20 seconds in... let's see if it sticks.
<wgrant> Apparently not.
<wgrant> Is there another build appearing in the logs now?
<wgrant> Ah.
<wgrant> The other two builds are now at the head of the queue.
<elmo> 2010-08-07 23:50:38+0100 [-] startBuild(Maverick, daily, gnome-terminator (on http://muntries.ppa:8221/))
<elmo> 2010-08-07 23:50:38+0100 [-] Scanning failed with: 'NoneType' object has no attribute 'email'
<wgrant> https://code.edge.launchpad.net/~gnome-terminator/+recipe/daily/+build/516 and https://code.edge.launchpad.net/~gnome-terminator/+recipe/daily/+build/517 need destruction.
<wgrant> Yep.
<elmo> done
<wgrant> 40 seconds in. Looking good.
<elmo> I'm tailing the log looking for noe
<elmo> noNe
<elmo> will this happen again tomorrow?
<wgrant> Possibly.
<elmo> awesome
<wgrant> The API won't tell me who requested them :(
<wgrant> Hmm.
<wgrant> I guess it probably uses the recipe owner for daily builds.
<wgrant> So it might be a good idea to disable daily builds of that recipe for now, or it will indeed happen again tomorrow.
<elmo> do you know the url for that offhand?
<elmo> or even pointers to it
<wgrant> https://code.edge.launchpad.net/~gnome-terminator/+recipe/daily/+edit
<wgrant> There should be a checkbox.
<elmo> WTF
<elmo> that says *I'm* the owner
<wgrant> Oh no, not that bug...
<elmo> that's either an awesome troll by Ng or an epic UI bug in LP
<wgrant> It's a known bug, yes.
<wgrant> Either find someone in the team to do it, or just twiddle the flag manually.
<elmo> manually == SQL?
<wgrant> Yeah.
 * elmo rolls his eyes
<wgrant> Sorry :(
<elmo> it's not your code
<elmo> (I assume)
<wgrant> Heh, no.
<wgrant> And we have logtails... so everything's working again for now. Thanks.
#launchpad-dev 2010-08-08
<elmo> ok, I'm wandering off again, available on the phone if it breaks more/again/harder
<wgrant> Night.
 * wgrant files a bug.
<lifeless> elmo: thanks
<lifeless> wgrant: why would teams need a mail address ;P
<wgrant> Bug #614858
<_mup_> Bug #614858: Recipe builds fail when requester does not have a preferred email address <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/614858>
<wgrant> lifeless: Well, they often do (preferredemail is the contact address).
<lifeless> wgrant: fvso often
<wgrant> lifeless: True.
<lifeless> I'd want to do a stats run on that before thinking about it
<lifeless> Ursinha-afk: / matsubara-afk: are either of you around ?
<jelmer> 'morning wgrant, lifeless
<lifeless> hai
<lifeless> jelmer: have you considered sleeping ?
<jelmer> lifeless: what's that, "sleep" ?
<lifeless> its the root for sleeping.
<lifeless> :P
<wgrant> elmo: When you wake up... the world has collapsed again.
<wgrant> I can't see what's done it, but it's not the same recipe.
<cody-somerville> wgrant, Lucid, lmsensors-trunk, team-iquik
<wgrant> cody-somerville: Sigh.
<wgrant> ...........................
<wgrant> What the *fuck*.
<wgrant> Seriously guys.
<wgrant> Unbelievable.
<wgrant> Anyway, everything's working again.
<wgrant> Bah, I think things are broken again.
<wgrant> Yes :(
<lifeless> details?
<wgrant> I can't tell.
<wgrant> Needs log access.
<elmo> 2010-08-08 09:10:06+0100 [-] startBuild(Lucid, dailylucid, nova-core (on http://thorium.ppa:8221/))
<elmo> 2010-08-08 09:10:06+0100 [-] Scanning failed with: 'NoneType' object has no attribute 'email'
 * wgrant kills it dead.
<wgrant> I wonder why this is only happening now.
<wgrant> Maybe dailies weren't being created until recently.
<wgrant> Oh!
<wgrant> They would have never reached the head of the queue until recently, because of the queue fuckup over the last couple of weeks.
<thumper> elmo: can just disable the daily build script until we fix it?
<elmo> thumper: if you give me some hint as to how to do so, sure?
 * thumper loks
<wgrant> It's request_daily_build.py, but NFI where it runs.
<wgrant> request_daily_builds.py, sorry.
<elmo> not cesium
<elmo> so it must be loganberry
<elmo> checking
<wgrant> Yeah.
<thumper> request_daily_builds.py
<thumper> loganberry
<thumper> possibly as bzrsyncd
<elmo> yeah, it is. ok, commenting out it and the check for it
<thumper> elmo: ack, thanks
<wgrant> Thanks.
<thumper> I'll take an urgent look tomorrow morning
<thumper> wgrant: is there a bug?
<wgrant> thumper: For one of the issues: bug #614858.
<_mup_> Bug #614858: Recipe builds break the world when requester does not have a preferred email address <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/614858>
<thumper> ta
<wgrant> For the others which I discovered a couple of hours ago... not yet.
<thumper> what was the other issue?
<elmo> ok, all done
<elmo> (disabling)
<elmo> 2010-08-08 09:18:30+0100 [-] startBuild(Maverick, rekonq-daily, rekonq (on http://lychee.ppa:8221/))
<wgrant> I wonder how many there are left.
<wgrant> Probably lots.
<thumper> there are quite a few daily recipies now with team owners
<wgrant> Shall we play suspend-all-builds-without-a-preferred-email-address?
<elmo> I'm game if it involves SQL and not me tailing a log and pasting broken ones to the channel
<wgrant> That's what I was thinking.
<elmo> (esp. since I need to get on a plane at some point today)
 * wgrant SQLs.
<wgrant> Ah, IS sprint?
<elmo> wgrant: ta
<elmo> wgrant: yep
 * wgrant waits for download-cache to update...
<wgrant> elmo: Ahem. One query for the wrong (10.08) schema later, http://paste.ubuntu.com/474867/ appears to work fine.
<wgrant> Ok.
<wgrant> #launchpad intrigues me.
<wgrant> How has this error come up so many times today?
<lifeless> interesting
<lifeless> possibly things to check
<lifeless> a change in sql for something in registry?
<lifeless> a cp of a soyuz change on friday?
<lifeless> something else?
<lifeless> like latency for backlog catchup
<elmo> wgrant: thanks; there was only actually 3
<lifeless> \o/ am up to actual new functional in qa-tagger refactoring.
<lifeless> OTOH mars and ursula are going to kill me Monday :)
<lifeless> \o/ success
<lifeless> jml: did I see you just come online ?
<jml> lifeless, maybe
<jml> lifeless, wassup
<lifeless> I want to sketch a fixtures thing
<lifeless> if you're up for a little voice - very little - on that that would be nice
<jml> sure.
<jml> but I need coffee first
<jml> ~!5
<jml> ~15m
<lifeless> ok
<lifeless> well, ping me when you're back
<lifeless> if I'm up, we'll chat
<lifeless> if not, it'll brew eventually anyhow :)
<jml> hi
<jml> lifeless, ping
<lifeless> hi
<Ursinha-afk> is this actually monday for you guys or you're just having fun? :)
<neo_> echo hi
<james_w> lifeless: you might be interested to look at https://code.edge.launchpad.net/~james-w/launchpad/more-matchers/+merge/32057 where I felt Matchers didn't support me too well in what I wanted to do. I filed a bug on one thing that would help. It may partly be that keeping the existing assert* API didn't help.
<lifeless> +        except NotFoundError:
<lifeless> +            return self.new(name)
<lifeless> james_w: is new an instance method ?
<james_w> lifeless: I don't think I wrote that?
<james_w> or maybe that's in the prerequisite branch, which got included in this diff to start with.
<lifeless> its in the diff for https://code.launchpad.net/~james-w/launchpad/more-matchers/+merge/32057 that I got mailed to me ;)
<james_w> right, again only mailing the first diff wastes people's time
<lifeless> james_w: where was this bug you filed ?
<james_w> lifeless: https://bugs.edge.launchpad.net/testtools/+bug/615108
<_mup_> Bug #615108: Please provide a way to match an iterable of matchees <testtools:New> <https://launchpad.net/bugs/615108>
<thumper> morning
<lifeless> james_w: MatchesMany ?
<lifeless> james_w: or 'IsIterableOf
<lifeless> bbiab -> allergy shot
<wgrant> I can't seem to access MPs on staging.
<wgrant> OOPS-1681S1581 is a timeout, and OOPS-1681S1580 is an actual OOPS.
<thumper> wgrant: is it always timing out?
<thumper> it could be a staging librarian issue
<wgrant> One times out, the others OOPS.
<wgrant> The OOPS would make sense if the diff was gone.
<thumper> wgrant: is there an oops for bug 614858 ?
<_mup_> Bug #614858: Recipe builds break the world when requester does not have a preferred email address <Launchpad Bazaar Integration:Triaged by thumper> <https://launchpad.net/bugs/614858>
<wgrant> But timing out as well on others is a little odd.
<wgrant> thumper: No OOPS.
<wgrant> Just stuff in buildd-manager logs.
<wgrant> And it doesn't do OOPSes (yet?)
 * thumper sighs
<wgrant> Why?
<wgrant> The problem is obvious.
<thumper> wgrant: I don't know the buildd code as well as you
<thumper> wgrant: where do you think the exact issue is?
<wgrant> thumper: lib/lp/code/model/recipebuilder.py -- search for 'preferredemail'
<thumper> heh
<wgrant> Ahem, oops.
<thumper> ok, I'll admit to have not worked on any real recipe stuff
<jelmer_> wgrant: I'm surprised it is possible for a user to not have a preferredemail address, when does that happen?
<wgrant> jelmer_: They're deactivated, or it's a team without a contact address.
<jelmer_> (I vaguely remember writing that code in Wellington)
<wgrant> jelmer_: And a team 'requests' a recipe build if they are the owner of a daily recipe.
<jelmer_> wgrant: Why would a deactivated account not have a preferredemail ?
<jelmer_> is that removed when they are deactivateD?
<thumper> I think a deactivated person still has a preferred email
<thumper> the problem here is a recipe owned by a team
<wgrant> jelmer_, thumper: Deactivation resets the preferredemail to NEW.
<wgrant> So yes, the person ends up without a preferredemail.
<wgrant> This may be how mail to deactivated users is prevented.
<wgrant> (See the end of Person.deactivateAccount)
<jelmer_> wgrant: Thanks
<wgrant> thumper: bug #615144
<thumper> wgrant: ta
 * thumper afk to get the car a warrent
 * jelmer waves goodnight
#launchpad-dev 2011-08-01
<poolie> wgrant: should i kill it in ec2?
<wgrant> Let's see what version of python-apt is on pigeonpea/pilinut/cocoplum/germanium/cesium.
<wgrant> LOSAs: ^^
<hloeung> wgrant: one sec
<hloeung> wgrant: 0.7.94.2ubuntu
<wgrant> There are three characters missing there.
<wgrant> And they are the critical ones :)
<hloeung> 0.7.94.2ubuntu6.2, does that sound right?
<wgrant> That sounds right.
<wgrant> But also old :(
<wgrant> poolie: ^^ we need 6.4
<poolie> ok
<poolie> hloeung: do those machines not get l-updates automatically?
<poolie> well, i realize approximately no machines upgrade really automatically
<poolie> but would it be hard to bring it in? is it seen as being available?
<hloeung> poolie: yeah, it seems to be available.
<poolie> i killed the ec2 land run and i'm running just a test
<poolie> thanks haw; how would we go about getting it upgraded?
<poolie> a request on the lp production status page?
<hloeung> file an RT about it
<hloeung> and we'll do it for you
<poolie> lifeless: should i/we do this ^^?
<lifeless> poolie: I'd rather someone involved with the bug do it
<lifeless> the side effects of an uncoordinated change can be severe
<poolie> mm
<poolie> who would that be? jelmer?
<lifeless> jelmer, wgrant, bigjools, gavin, stevenk, jtv or colin - offhand those folk should all have the experience to make reasonable predictions about side effects
<poolie> cool
<poolie> i'm pushing it because it has been mentioned as a udd blocker
<lifeless> so basically the branch can't land until we've separately done the upgrades everywhere
<lifeless> and that needs teseting and QA in its own right.
<lifeless> e.g. start with buildbot / *staging, dogfood, check its all good, then prod, then and only then land the branch.
<poolie> yep, thanks
<wgrant> poolie: A UDD blocker!?
<poolie> well, not a blocker for packaging branches, but a thing people complained about in our meeting
<wgrant> Interesting.
<wgrant> 'cause it's not really related at all...
<lifeless> poolie: what trouble was it causing them ?
<poolie> i believe it was that they wanted to sync tar.xz packages from debian
<poolie> ?
<poolie> the comment was basically just "can we have that xz bug fixed?"
<lifeless> heh
<lifeless> well, I'm glad you have an interest in it
<lifeless> it will be good to have it fixed
<poolie> .. and i just hate to see thing stalled, at least without a "this is waiting on" comment
<poolie> lifeless: do you want to read https://code.launchpad.net/~mbp/launchpad/mail-scope/+merge/60281 again?
<poolie> it has a +1
<lifeless> sure, i can do that
<StevenK> Has anyone seen http://paste.ubuntu.com/655990/ before?
<poolie> i think i've seen something like that, where _testMethodName is missing
<poolie> as the name implies its failing to load and then things are not sufficiently set up to report the faliure
<poolie> i'd suspect a syntax error or similar
<poolie> or an import erorr
<StevenK> pyflakes is happy, which is the odd thing
<poolie> i would probably bisect backwards to find something that works
<StevenK> If I comment out a class variable "foo = PillarName.projectgroup == None" it works
<poolie> ah, well maybe that's stopping it loading
<poolie> perhaps projectgroup is not defined at the time that initializer runs?
<StevenK> Hmmmm, maybe it isn't projectgroup
<poolie> lifeless: shall i wait for you to review the mail_header patch or send it now?
<StevenK> Rargh!
<StevenK> PillarName has product and project
<StevenK> Which one is projectgroup, then?
<wgrant> project
<wgrant> Product = Project, Project = ProjectGroup
<lifeless> poolie: I have reviewed it now; I had to pop out to the vet before.
<poolie> np, thanks
<lifeless> poolie: I think its fine as is but would be smaller if you derive from Fixture
<StevenK> wgrant: That's ... obvious :-/
<poolie> lifeless: oh, interesting idea, even though it's not used only in testing?
<lifeless> fixtures isn't just for testing
<lifeless> never was
<poolie> ok
<nigelb> wallyworld: hi
<poolie> o/ nigelb
<nigelb> hey poolie!
<wallyworld> nigelb: hello
<wallyworld> how's the branch going?
<nigelb> wallyworld: I was working on solving that bug last weekend and ran into doubts.  I emailed you :)
<nigelb> ..and the branch is here -> https://code.launchpad.net/~nigelbabu/launchpad/4595-upgrade-bug-linking
<wallyworld> nigelb: really? i don't recall seeing the email
<wallyworld> let me check my inbox
<nigelb> :)
<wallyworld> nigelb: it was in my trash - i got a whole lot of code import email which escaped my filter arrive in my inbox and yours was right in the middle and accidentally got deleted
<nigelb> haha
<wallyworld> give me a few minutes to read it :-)
<nigelb> oh. 240 critical bugs? :(
<wallyworld> nigelb: so i see you've added the 'bug-link' class to bug links and are grabbing those to send to the back end
<nigelb> Yeah.
<nigelb> I got that far. Now at the javascript is where I have a few doubts on how to proceed
<wallyworld> did you have a particular question?
<nigelb> Yes, so in the javascript.
<nigelb> I'm doing harvest_links(links_to_check, 'branch-short-link', 'branch_links', 'bug-link');
<nigelb> Is that the right way, just add the css for the bug along with the branches
<nigelb> I was wondering if they should be seprate variables
<wallyworld> the server has one end point for processing any links extracted from the page, so it makes sense just to add a new link class to the existing variable
<wallyworld> ie gather all links, send them to the server for processing, render the results
<nigelb> ah, so now I have to go to the server and add stuff to the processing bit.
<wallyworld> on the server side you can introduce a separate class for bugs vs branch links
<nigelb> ah, ok
<wallyworld> the reason it can be in the one data structure is that the server is just providing generic info (enabled, title text etc)
<wallyworld> so on the client, there's no reason to distinguish between bug vs branch links
<wallyworld> does that make sense?
<nigelb> Yes, it does :)
<wallyworld> excellent!
<nigelb> I guess the next place I should be making modifications is lib/lp/app/browser/linkchecker.py
<wallyworld> yes
<wallyworld> or you could write some tests for the javascript side ie TDD
<wallyworld> get that side of things sorted first
<nigelb> ah, yes.
<wallyworld> there may not be yui tests for that stuff
<nigelb> there isn't.
<wallyworld> the framework we have now was not around back when it was written
<nigelb> so I guess I'll have to write new tests
<wallyworld> yeah, that would be good if you had the time
<nigelb> I'll try :)
<nigelb> Not sure if I can get them rright.
<wallyworld> or at least write tests for the stuff you are adding
<wallyworld> i can help
<nigelb> I'll push to the branch tonight and pke you tomorrow to see if I've done it right
<wallyworld> nigelb: you could look at a simple existing test and use that to get started eg test_hide_comment.js
<nigelb> I did look at one
<nigelb> It looks easy enough.
<wallyworld> also http://dev.launchpad.net/JavascriptUnitTesting
<nigelb> (famous last words, I know :P)
<wallyworld> yep :-)
<nigelb> hehe, who wrote that wiki page?
<nigelb> "Day one was horrible, there was a lot of shouting, words were said that may have hurt my computer's feelings"
<wallyworld> not sure, but that's funny :-)
<nigelb> yeah, heh
<nigelb> Interesting.
<nigelb> make run now takes only ~30s
<nigelb> Laters! Off to work.
<wallyworld> bye
<jtv> StevenK, wgrant, greetings!  Continuing Q/A on the python-based publish-distro script.
<wgrant> jtv: While we are there, it seems that publish-distro.py itself is now causing ~1500 OOPSes a day, because each WARNING spewed by CommandSpawner from a-f stderr is an OOPS now :/
<jtv> wgrant: I imagine a lot of that simply shouldn't have been WARNING though.
<jtv> That's assuming that this is mainly increased urgency & visibility for problems that were previously just as big.  Or have substantial new problems cropped up?
<wgrant> jtv: apt-ftparchive's stderr output is... verbose.
<StevenK> And 65 OOPSes per publisher run is just oh so much noise
<wgrant> But the OutputLineHandler we give to CommandSpawner is self.log.warning
<wgrant> Which, in a LaunchpadCronScript, causes an OOPS.
<jtv> Then that should probably just change.
<wgrant> Indeed.
<wgrant> Well.
<wgrant> Ideally we would tell a-f not to spam crap to stderr.
<jtv> Quite.
<wgrant> But that sounds more difficult than s/warning/info/
<jtv> Well, be the first kid on your block to fix a thousand oopses today this week!
<wgrant> And not hugely more rewarding.
<wgrant> I'm not allowed to :)
<wgrant> But maybe.
<jtv> Borrow abentley's certificate.
<wgrant> Heh.
<jtv> If it's Critical and gets in the way of normal workâ¦
<jtv> (Which would equally be an excuse for me, so I could pick it up if you can't)
<wgrant> Critical isn't an excuse for you.
<wgrant> Being in scripts that you're working on is.
<jtv> I was thinking of the "gets in the way of normal work" part, but yes.
<jtv> Meanwhile, I need to upload test packages to the security pocket & a partner archive for further publish-ftparchive testing.
<wgrant> Fun.
<jtv> (Have you ever used the python version?)
<wgrant> You can't build in security directly, so you have to copy to elsewhere.
<wgrant> s/to/from/
<jtv> Ah
<jtv> that un-confuses me.  :)
<jtv> And partner?
<wgrant> partner you just upload with a component of partner.
<wgrant> (that is Section: partner/SOMESECTION)
<jtv> That's in the .dsc?
<wgrant> Right.
<wgrant> Well.
<StevenK> And debian/control in the unbuilt
<wgrant> Set it in debian/control.
<wgrant> dpkg-buildpackage will put it in the .dsc
<wgrant> If you edit the .dsc directly, there will be great sins to atone for.
<StevenK> How should I search for truth in Storm? PillarName.active is True or PillarName.active == True ?
<StevenK> Although I seem to get inactive results in either case
<lifeless> ==True is what most of our code does
<lifeless> this isn't good SQL
<lifeless> but that  should be changed in storm
<StevenK> I'm still concerned I'm getting inactive results when both inner-selects have PillarName.active == True
<lifeless> whats the full query?
<StevenK> lifeless: http://pastebin.ubuntu.com/656105/
<lifeless> StevenK: why the ALL ?
<StevenK> Without the ALL it would not return the rank 100 result sometimes
<lifeless> the implicit distinct on a UNION is on all the columns
<lifeless> I suspect a bug in your _filter - it could explain both situations (the current confusion and the rank 100 missing sometimes)
<StevenK> lifeless: I have two subclasses, for one of them I want a filter
<StevenK> lifeless: If you want the actual query that the code produces, I can get that for one of the cases
<lifeless> StevenK: I'd like to see the actual query postgresql sees
<StevenK> Yes, I was going to get the query from postgres's log
<lifeless> StevenK: (or run with LP_SQL_DEBUG=1)
<wgrant> StevenK: 'is True' can't work, '== True' does.
<StevenK> LP_DEBUG_SQL=1 and doctests suck
<jtv> Hah.  System death.  Just what I needed on the day I discovered I left my power supply in Europe.
<StevenK> lifeless: http://pastebin.ubuntu.com/656113/
<StevenK> lifeless: I don't like the look of the "AND NULL"
<lifeless> thats your _filter; and yes, thats a little worrying ;)
<StevenK> lifeless: I only need to filter for one case -- I suspect it is None for the other case
<lifeless> a better way to construct things would be a list
<lifeless> And(*clauses) [or skip the and, find's default join is AND
<StevenK> lifeless: That looks like a better query. Sort of
<StevenK> .contains_string() is just shit SQL
<lifeless> you should be able to drop the ALL too
<lifeless> (unless you specifically want it)
<StevenK> This query is based on one that Curtis wrote for me, and that used UNION ALL
<jtv> lifeless: the ALL in a UNION is usually something you want to keep if possible.
<lifeless> jtv: why?
<jtv> Saves a filtering pass.
<jtv> (And thus, potentially a sorting pass)
<lifeless> jtv: only if its unsorted though?
<lifeless> jtv: and all our vocabs are sorted
<jtv> It depends on the details, which I don't know and prefer not to get into right now; what matters is the uniqueness of entire rows, and chances are that the planner can't determine statically that that's not an issue.
<lifeless> sure, thanks for mentioning this
<jtv> (Chances are that it could in theory, but for it to try might significantly increase the run time of very fast and very frequent queries)
<jtv> np
<jtv> StevenK: what was the fs root for process-upload again?
<wgrant> /srv/launchpad.net/ubuntu-queue
<jtv> thx
<jtv> Can't we make this a configuration item at some point?
<wgrant> Not really. There are several queues in active use.
<wgrant> The two poppy queues (there were four, now only two), the buildd-manager queue, the sync queue, the backdoor queue.
 * jtv sobs quietly
<jtv> "why does it all have to be so _hard_?"
<jtv> Oh for Kibo's sakeâ¦ battery critical already!?
<wgrant> Hmm.
<wgrant> Is lib/canonical/launchpad/webapp/templates/launchpad-model.pt used?
<wgrant> It was added in r13548 and is buggy.
<wgrant> But grep shows nothing.
<rvba> Good morning.
<adeuring> good morning
<henninge> moin adeuring!
<adeuring> hi henninge!
* henninge changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer:  henninge | Critical bugs: 240 - 0:[#######=]:256
<stub> We are going to need a report on what db patches have been applied to (production, staging, qastaging) vs. what have landed in what branch.
<stub> Is there a smart way of seeing what files exist in the database/schema directory of lp:~launchpad-pqm/launchpad/db-devel besides checking out the tree?
<StevenK> loggerhead
<bigjools> StevenK: bzr ls
<bigjools> StevenK: bzr ls
<StevenK> Or bzr ls
<bigjools> argh
<StevenK> bigjools: FAIL
<bigjools> stub: bzr ls
 * bigjools kicks auto-complete in the spuds
<stub> Seems to be downloading a heap, but that likely doesn't matter on the LAN.
<stub> 1.7MB and counting...
<stub> 4MB...
<stub> Is this going to download the whole tree?
<lifeless> hmm, it would be good for the qatagger to do this, roll it into the same report
<lifeless> stub: no
<lifeless> stub: but it will be downloading pack data, and if you are behind a broken http proxy this can get quite scattered
<stub> Ooh... this is ssh. Wonder if HTTP goes faster locally for this sort of thing?
<stub> finished now. should be fine on the lan. ta.
<stub> And using the recursive option might help
<lifeless> stub: or just ls database/schema
<stub> bzr ls lp:~launchpad-pqm/launchpad/db-devel was what I first did, which strangely enough got me the top level directory listing.
<lifeless> strange that
<stub> Is there anything stopping bzr on devpad being upgraded? Missing -d, -k on that command.
<stub>  bzr ls -v lp:~launchpad-pqm/launchpad/db-devel/database/schema seems to do what I want with the old version
<stub> If course, if I want to show revisions changed as well I should probably jump in and use bzrlib rather than try to parse the output of bzr log
<lifeless> nothing stopping it being upgraded
<lifeless> stub: I'd seriously consider patching the qatagger for this
<lifeless> stub: the only tricky bit would be knowing what patches are live; perhaps a query page for that.
<stub> lifeless: That could be a good hook.
<lifeless> stub: why does the garbo merge depend on the bugsummary rollup peformance patch?
<stub> I was considering a tabular report   db | db | db | branch | branch , with the db columns boolean and the branch columns a list of revisions the patch was touched in.
<stub> But maybe this is better solved with similar process like qa-foo tags?
<lifeless> stub: what are we solving ? [for clarity, rather than me guessing :P]
<stub> lifeless: Because the db patch not only attempted to optimize the stored procedure, it added an optional 'batchsize' argument to the rollup function
<lifeless> stub: ah
<stub> lifeless: juggling merges of code that depend on database patches being applied
<stub> lifeless: nothing urgent in there, so I've just added that branch to my todo list. With frequent merges, it shouldn't be a problem
<lifeless> stub: so, http://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-devel/revision/10832 looks like a hot patch
<stub> It is, and I will be applying it live shortly with one of deryks, but the test suite won't pass for the garbo branch without that patch in the tree.
<lifeless> cool
<lifeless> so what I've been recommending folk do is get the patch live and then land the patch to devel, and then dependent patches
<lifeless> this depends on us getting the latency of application down
<lifeless> but avoids lots of cherrypicks and hard-to-qa stuff in db-devel
<lifeless> perhaps this is wrong
<lifeless> stub: so the report; uhm; I would hope that any code which /needs/ a db patch would fail tests without it.
<lifeless> stub: I think a report is very useful to have.
<stub> lifeless: agreed on both
<lifeless> devs probably can look in a source tree easily enough for something they care about
<lifeless> the thing devs /cannot/ do today is determine whether a patch is live without nagging a losa / gsa/ dba
<wgrant> Should we just have a trivial view which returns the applied patches?
<stub> yer, the important thing is what patches are on what db (which could be done with a view in Launchpad)
<stub> snap
<stub> The report would present all that info in one place though, as well as any interesting trivial like a db patch that was modified after landing in a particular tree.
<wgrant> Sounds like it might belong in the deployment dashboard.
<lifeless> well, thats why I was suggesting qatagger.
<wgrant> Mmm.
<lifeless> unapplied db patches
<wgrant> I'd prefer another data provider that lpqateam.canonical.com watches.
<wgrant> Rather than forcing it into qa-tagger.
<wgrant> It doesn't seem to be a natural fit.
 * lifeless shrugs
<lifeless> the biggest ROI seems to be on a view for what-patches-are-live-on-this-server
<bigjools> wgrant: can we talk about initialisation checks now?
<bigjools> that scared him off
<wgrant> bigjools: Hi.
<wgrant> bigjools: http://jam-bazaar.blogspot.com/2009/11/memory-debugging-with-meliae.html
<danilos> henninge, hi, do you mind reviewing https://code.launchpad.net/~danilo/launchpad/bug-814580-pre-cleanup/+merge/69980?
<henninge> wgrant: Hi!
<henninge> wgrant: "Needs information" on https://code.launchpad.net/~wgrant/launchpad/iseriesbugtarget/+merge/69951
<henninge> danilos: sure
<danilos> stub, lifeless: I've got two DB patches up for your review, one is "hot", another is "cold" (and I don't care about getting that one done "soon", since it's a table rename)
<danilos> henninge, thanks
<henninge> danilos: Relative packages in zcml? Are we doing that nowadays?
<wgrant> henninge: It needs a bug?
<henninge> wgrant: well, some information about the motivation for the chage.
<wgrant> Indeed, but not a bug :)
<henninge> wgrant: there must be some code in use (or in future use) where this will be used.
<henninge> wgrant: yeah, probably, because it needs no qa.
<wgrant> Well, there already is one case: shouldIndentTask. I thought the utility was fairly clear, regardless of future specifics. But I shall update the MP with the real intent.
<danilos> henninge, there's a bunch of that in existing .zcml, and this makes it easier to adhere to our line length limits (though, it makes it harder to grep for stuff)
<danilos> henninge, I am not too attached to the formatting so if you want, I can make it complete
<henninge> danilos: I was just wondering. You can leave that as it is.
<henninge> danilos: lines 255, 261, 303: I'd do full indentations in that case.
<danilos> henninge, what do you mean? 4 spaces? (this is how Emacs python-mode does it by default)
<henninge> danilos: also, shouldn't test_translationpackagingjob be renamed, too?
<danilos> henninge, translationpackagingjob is not renamed
<danilos> henninge, ideally, it should
<danilos> henninge, or well, not, because that one is for the Packaging links being introduced/removed
<danilos> (it should be renamed in my follow-up branches where I add the TemplateChangeJob)
<henninge> oh, I mixed that up.
<henninge> danilos: never mind about the indentations, they work for me the way they are.
<henninge> danilos: just needed something to complain about ...
<henninge> danilos: r=me ;-)
<danilos> henninge, excellent, thanks
<danilos> henninge, btw, how familiar are you with zope events and testing them? (basically, I wonder about testing IObjectModifiedEvent on potemplate, and the event doesn't fire if I just set potemplate.name to something)
<henninge> danilos: Zope what? ;-)
<henninge> danilos: no, I am not familiar with th em
<danilos> henninge, ack :)
<wgrant> danilos: You need to create a Snapshot and then notify() the ObjectModifiedEvent manually, unfortunately.
<danilos> wgrant: really? what I find weird is that IObjectCreatedEvent and IObjectRemovedEvent are tested in the same file (albeit on a different object) and they work just fine; is this specific to the Modified event?
<wgrant> danilos: Those two are easy.
<wgrant> There are no field changes to take care of.
<wgrant> ObjectModifiedEvent has to diff the objects.
<danilos> wgrant: true, ok, just knowing that's what I've got to do is helpful enough, it kinda felt like cheating
<jml> if you use dput to upload to a PPA that doesn't exist, when do you get the error message?
<wgrant> jml: An email after 0-5 minutes.
<jml> wgrant: thanks
<wgrant> Yes, yes, this sucks :)
<jml> And, IIRC, making it not suck requires message queues and some nebulous amount of rejiggering.
<bigjools> it could do that check in poppy along with the GPG check
<wgrant> To do it properly, yeah.
<wgrant> bigjools: The amount of DB access it has now is already problematic and excessive :(
<wgrant> We will have a message queue within months.
<bigjools> ??? it's tiny
<jml> for my part, I could *probably* get away with LBYL.
<wgrant> And the world will be a better place.
<wgrant> bigjools: Yes, but it's existent.
<wgrant> bigjools: This makes DB upgrades harder, makes it much more fragile, leaves it stuck in our tree...
<danilos> henninge, there's another one I need reviewed (just need to fix this problem with event firing for a single test, rest is all good): https://code.launchpad.net/~danilo/launchpad/bug-814580/+merge/69978
<henninge> danilos: don't you have any shorter ones? ;-P
<gary_poster> thanks for review henninge :-)
<henninge> gary_poster: you are welcome
<danilos> henninge, heh, I do but they are DB patches :P
<henninge> danilos: ah, no thanks ;-)
<henninge> danilos: I'll just do this one, then ... ;-)
<danilos> thanks again :)
<henninge> danilos: I remember that this kind of change used to be a DB change. Did that change? ;)
<henninge> http://paste.ubuntu.com/656349/
<henninge> ETOOMANYCHANGE
<danilos> henninge, adding a new value to the enumeration isn't :)
<wgrant> Well, it has some similar consequences.
<wgrant> You have to be sure that various components of the system will cope with not knowing about it.
<wgrant> Which will normally make stuff OOPS.
<danilos> wgrant: right, that has been accounted for
<henninge> danilos: ok, if you say so. Don't blame me if it can't be merged .... ;-)
<wgrant> It's mergable, but it might break production :)
<danilos> henninge, heh, fair enough
<henninge> well, that's not that bad, then.
<danilos> :)
<henninge> :)
<danilos> nah, this is a translation packaging job type, so if it breaks anything, it's going to break that, and that needs fixing anyway
<henninge> I once tried that and PQM wouldn't merge it into devel. (I think that's where it failed)
<henninge> but that is > 18 months ago.
<danilos> henninge, wow, weird
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: henninge, benji | Critical bugs: 240 - 0:[#######=]:256
<wgrant> jml: Are you able to QA your PPA API thingy?
<jml> wgrant: I'm in the middle of something right now. you lining up for a deploy?
<wgrant> jml: Checking up on EU/US QA so I don't have to do much tomorrow, since I'm not allowed to.
<jml> wgrant: essentially, I can do it, provided someone else can grant & revoke commercial admin privs for me, but I would rather not.
<wgrant> k, I might just do it tomorrow, if nobody does it in the meantime.
<wgrant> Since we are 30 revs behind now :/
<jml> yikes.
<bigjools> wgrant: is this the create-as-private thing?
<wgrant> bigjools: Ja.
<bigjools> wgrant, jml: qa-bad I think.  My private=False parameter is ignored.
<wgrant> I actually don't see any logic that sets private = True...
<StevenK> The whole point is to call it via the API
<wgrant> I mean, there is nothing in createPPA that uses the argument.
<bigjools> ...
<wgrant> Except for validatePPA.
<bigjools> yup
<wgrant> bigjools: Can you roll it back, please?
<bigjools> I'm sure a maintenance team can do it, I'm kinda busy
<bigjools> anyway it can roll out, it's not a blocker
<wgrant> Mmm. It's fairly misleading to have a 'private' argument that does nothing.
<bigjools> affects like 7 people
<wgrant> But I guess the set of affected users is small.
<wgrant> Yes.
<bigjools> oddly createPPA is only exported on version=beta
<StevenK> If it doesn't block deployment then file a bug saying it doesn't work and move on?
<bigjools> just re-open the old one
<bigjools> mark it qa-ok
<cody-somerville> wgrant, This is probably a stupid question, do you have the necessary permissions to set it private?
<wgrant> cody-somerville: No.
<bigjools> I do, and I tested it
<StevenK> cody-somerville: However, wgrant did state the code didn't *do* anything with the flag.
 * bigjools wonders what the tests look like for that change
<StevenK> I think there is a validatePPA test only
<wgrant> [r=julian-edwards] :P
<StevenK> And no end-to-end test that I recall
<bigjools> wgrant: :(
<bigjools> not one of my best reviews
<StevenK> Don't do that
 * wgrant sleeps.
<henninge> abentley: bug 819241
<_mup_> Bug #819241: TranslationMergeJob migrates only POTMsgSets and no translations <upstream-translations-sharing> <Launchpad itself:Triaged> < https://launchpad.net/bugs/819241 >
<henninge> abentley: bug 814580
<_mup_> Bug #814580: Merge/split messages from all sharing templates <translations-handoff> <Launchpad itself:In Progress by danilo> < https://launchpad.net/bugs/814580 >
<abentley> gary_poster: about the jsoncache, I didn't understand what you meant by : My preferred solutions might be long poll pushes, and/or MVC in client-side code (which in isolation would only address part of the "curentness" problem, but would be better than what we have)
<abentley> gary_poster: how would MVC help?
<gary_poster> abentley, right.  MVC in isolation--by which I mean edits change a model, and the model is updated from the server in isolation, and any connected model updates need to be explicitly connected/requested--gives you some sanity within a given page.  Multiple displays based on the same data will agree.
<gary_poster> This will be sufficient in N% cases, where I suspect N is relatively high; and it will involve calculating a lot less data.  The tradeoff is full page correctness/timeliness with indiscriminate data loads (jsoncache + an update mechanism such as MVC) vs. possibly good-enough correctness and less cost of gathering data with specifically requested data loads (MVC "in isolation").
<abentley> gary_poster: If "connected model updates need to be explicitly connected/requested", the only way to do that in our current web service is with one roundtrip per object, which doesn't give good performance.
<gary_poster> abentley, sure.  you are building a new mechanism, though, so a new mechanism could be built.  And so far a single model update has been sufficient for the (limited) cases I've been involved with.
<abentley> gary_poster: Okay, I thought you meant MVC could help without new mechanisms, which is what I didn't understand.
<gary_poster> abentley, it can and does, if your use case is sufficiently limited, as the ones I've encountered have been.  If there is merit in what I am saying, it is in the "80/20 balance" direction--that is, we usually don't need the cost of a full data reload
<abentley> gary_poster: From a user's perspective, I think that two roundtrips per operation (one to write, one to read) is the maximum we should tolerate, so partial updates will only be acceptable if only one or two objects are affected.
<gary_poster> abentley, sure.  Often writes include the new version of the affected object, though, which means you only need the write.  Anyway, I think you understand where I am coming from which I think was the goal of your question.  I don't need to convince you of my position.
<gary_poster> Hopefully what you have done will work out well, and if not, it will almost certainly be helpful in some cases at least.  If there are occasional performance problems, we can cross that bridge then, and if not, yay!
<abentley> gary_poster: Here's an option that might please both of us: During a write operation, use server-side events to generically detect changes to the object cache,  return a delta as the result of the write operation.  What do you think?
<gary_poster> abentley, in the abstract, yeah, I would like that--very nice.  ("in the abstract": IME we don't produce events reliably; but maybe we do it well enough that it would be nice functionality, and when it is insufficient, we can fix the event hookups.)
<abentley> gary_poster: Cool.  Yes, I expect trailblazers would have to verify that sufficient events were generated for their use cases.
<gary_poster> agreed abentley.
<gary_poster> that would be nice
<abentley> gary_poster: Can you think of a way to check whether all of our views are LaunchpadViews (or provide getCacheJSON)?
<gary_poster> abentley, I'm afriad you'd get a lot of false negatives, but you could use the component registry I believe.  I think it might give you a lot of views that are ones you don't care about for one reason or another, but maybe not.  Lemme see if I can get you a spelling...
<abentley> gary_poster: I'm okay with a lot of false negatives.
<danilos> henninge, how's review coming along? (I'd be the first to admit it's not the nicest branch, and that testing leaves a bit to be desired)
<henninge> danilos: sorry, distracted ...
<jml> bigjools, wgrant: huh.
<jml> thanks for the qa. I guess the test isn't doing what I thought it was.
<bigjools> jml: and I suck at reviewing :(
<jml> Oh. Duh.
<jml> Is that going to get reverted? Or am I ok to just patch what's in stable now?
<nigelb> bigjools: Didn't know you wwere big on cricket ;-)
<bigjools> nigelb: I don't like cricket.... oh no .... I love it-a
<bigjools> jml: it can land
<bigjools> patch away
<nigelb> bigjools: ha! shoulda poked you around world cup time ;)
<jml> ta
<bigjools> nigelb: who cares about the WC :)
<bigjools> 50-over cricket will die
<nigelb> T20WC? ;)
<nigelb> Though, I don't watch cricket much personally. Its all that place in our room's TV. Got some mad fans as roommates :)
<gary_poster> abentley: http://pastebin.ubuntu.com/656454/ .  There will be some duplication for the registratons of each layer.
<bigjools> nigelb: we are T20 champs :)
<abentley> gary_poster: Thanks very much.
<gary_poster> abentley, I forgot, you should run that within bin/py, and you should start out with
<gary_poster> >>> from canonical.launchpad import scripts
<gary_poster> >>> scripts.execute_zcml_for_scripts()
<gary_poster> yw
<nigelb> bigjools: wha? when? It was us once and Pakistan next.
<bigjools> nigelb: http://www.cricket20.com/db/t20_wc/default.asp
<gary_poster> http://pastebin.ubuntu.com/656458/ fwiw
<nigelb> bigjools: ha, I'm behind times apparnetly
<bigjools> nigelb: you may keep weeping
<nigelb> lol
<jml> https://code.launchpad.net/~jml/launchpad/create-private-ppa-814567-2/+merge/70028
<jml> would appreciate a review for that one
<jml> fixes a qa-bad
<jml> henninge: ^
<danilos> henninge, I am off now, so if you have any questions re review, they'll have to wait until tomorrow; cheers
<henninge> danilos: np, take care
* henninge changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: benji | Critical bugs: 240 - 0:[#######=]:256
<henninge> jml: r=me
<jml> henninge: thanks. can you please land it? I don't have commit privs.
<henninge> jml: sure
<henninge> jml: please set a commit message
<jml> henninge: done
<henninge> jml: I thought we had created a team for people like you. What was it called again?
<henninge> but I guess it does not have any special privs yet.
<jml> henninge: correct on both counts.
<jml> canonical-launchpad-emeritus, I think
<henninge> yes, that's what I meant
<henninge> jml: "Use --no-qa, or link the related bugs to the branch"?
<henninge> ec2 land wants a bug
<jml> henninge: the latter. the bug number is in the branch name.
 * henninge just figured that ... ;-)
<henninge> somtimes I type faster than I think
<benji> rvba: there are some conflict markers in https://code.launchpad.net/~rvb/launchpad/api-add-comment-bug-798522/+merge/70035
<rvba> benji: I just pushed a revision fixing the conflict.
<benji> cool
<adeuring> benji: could you have a look at this mp: https://code.launchpad.net/~adeuring/launchpad/bug-739052-2/+merge/70044 ?
<benji> adeuring: sure
<adeuring> thanks!
<jml> if I have a form where a user is giving me 'ppa:foo/bar', should I parse it myself, or is there something that'll do that for me?
<bigjools> jml: not heard of anything in LP but the dput code might
<jml> hmm. doesn't look like it.
<sinzui> jcsackett, ping
<jcsackett> sinzui: pong.
<sinzui> jcsackett, I have lost track of time. Do you have time to mumble?
<jcsackett> sinzui: certainly.
<benji> adeuring: I'm done with https://code.launchpad.net/~adeuring/launchpad/bug-739052-2/+merge/70044
<adeuring> benji: thanks!
<benji> my pleasure
<jcsackett> benji: could i add https://code.launchpad.net/~jcsackett/launchpad/privacy-notifications-on-branches/+merge/70050 to your queue?
<benji> jcsackett: sure; I'll look at it after lunch
<jcsackett> benji: thanks!
<jml> how best to check if a person is a member of a team over the API? Listing .participants is the best way I can see atm.
<dobey> jml: looping through the team memberships is how i do it. and you also have to loop through memberships of any of those memberships which is also a team
<dobey> becuase for some reaosn, someone though it was a brilliant idea to have team and person be the same thingâ¦ :)
<mtaylor> dobey: zope's internal design rears its head again
<jml> (team in list(person.super_teams)) seems to work
<dobey> jml: does that handle membership-by-proxy case?
<jml> dobey: docs say it traverses
<dobey> interesting
<jml> dobey: and it picks up a private team that I'm a member of via Landscape (of all things)
<dobey> jml: but only becuase you're using the API as you, so you can see that, right?
<jml> dobey: right. well, any member of that team could see it.
<dobey> jml: right. but that doesn't help my use case for tarmac :)
<jml> dobey: what conceivable use case could you have for seeing data that you're not allowed to see?
<jml> you can't see members of private teams unless you yourself are a member.
<dobey> jml: canonical is a private team, and all its members are not also members of the contributor-agreement team; and i have a tarmac plug-in that lets you check that people are legally valid contributors to a project
<jml> otherwise, they wouldn't be private
<dobey> and it's a pain that the bot user can't see the canonical team, since everyone in it can contribute
<jml> dobey: well, make the bot a member of canonical, or convince sinzui to change the privacy model
<dobey> well, the bot shouldn't be a member of canonical. it's a bot, not an employee. *shrug*
<dobey> and i'm not sure the privacy model should change
<sinzui> dobey, In a few months, we will have a mechanism to grant a user/team access to a project without placing the user in a role. This mechanism can and maybe added to teams. Private teams like their corporate owners to see what is happening inside them, and private teams like to subscribe non-members to archives
<dobey> sinzui: sounds interesting. all i really care about is doing team.is_member(person) :)
<LPCIBot> Project devel build #935: STILL FAILING in 5 hr 57 min: https://lpci.wedontsleep.org/job/devel/935/
<benji> jcsackett: I'm a little confused.  If branches already have a "private" attribute, why not mark them as providing IPrivacy?
<jcsackett> benji: i think the setup for the attribute in IBranch vs IPrivacy is different. honestly i read the comment where private is create in branch and decided not to tamper at this stage.
<jcsackett> one sec and i'll double check.
<jcsackett> benji: yeah, the IBranch interface defines it with several more parameters. is it safe to still mark branch as implementing IPrivacy, given IBranch sets it up differently?
 * jcsackett is not up on zope interface interactions.
<benji> jcsackett: parameters?
<jcsackett> benji: e.g. readonly=True
<benji> oh, are you talking about the web service API bits?  right, those are immaterial for in-process Python code
<jcsackett> benji: it's a parameter on the Bool declaration, which comes from zope.schema, right?
<benji> jcsackett: in the Liskov substitution sense a branch can't provide IPrivacy because IPrivacy says that "private" is read/write, so we have two... amongst our many options are: 1) be pragmatic, it'll work if we say branches provide IPrivacy and no kittens will be harmed or 2) make an (identity) adapter from IBranch to IPrivacy that just returns the branch
<benji> my vote would be for 1; the code will be simpler (no hasattr check) and it captures the meaning of what's going on
<benji> 2 would be a close second
<jcsackett> ok, i'll go with 1 and remove the changes to the formatter.
<benji> cool
<jcsackett> benji: turns out the problem is the branch is a "DecoratedBranch" which delegates Branch, so it has the private attribute, but doesn't convert to IPrivacy; i'm thinking i need to create an adapter for IDecoratedBranch to IPrivacy that just returns the Decorated branch, which as the private attr.
<benji> jcsackett: since the decorated branch already has a "private" attribute, why not make it provide IPrivacy?
<jcsackett> benji is that safe since it delegates to IBranch (which it turns out already *did* implement IPrivacy?
<jcsackett> if that's fine, i'll just do that; i always assume i'm going to break things when messing with this stuff.
<benji> be fearless
<benji> it should work fine, the object already has an attribute named what IPrivacy says it should be named and providing the information IPrivacy says it should provide (other than the read/write issue that we already discussed)
<jcsackett> benji: rock on.
<jcsackett> benji: just pushed changes up to LP.
<benji> k, I'll take a look
<allenap> Hi benji, got time for a medium sized uncontroversial branch?
<benji> allenap: sure
<allenap> benji: Thanks :) https://code.launchpad.net/~allenap/launchpad/selected-differences-vocab-bug-817408/+merge/70048
<benji> allenap: I'm done with https://code.launchpad.net/~allenap/launchpad/selected-differences-vocab-bug-817408/+merge/70048
<allenap> benji: Cheers.
<benji> jcsackett: I'm done with your's too (https://code.launchpad.net/~jcsackett/launchpad/privacy-notifications-on-branches/+merge/70050)
<jcsackett> thanks, benji!
<jcsackett> sinzui: when doing the new privacy notification for private teams, we're only adding it to the main team page, right?
<jcsackett> e.g. /~private-whatever/ but not /~private-whatever/+mugshots ?
<lifeless> morning
<lifeless> jcsackett: I would expect everywhere
<jcsackett> lifeless: possibly, there was some talk of "primary" context in discussions. i suppose the better question is what was meant by "primary context" with regards to private teams..?
<lifeless> jcsackett: so the question for you and sinzui is 'if someone is on <x page> and thinks its public, is it ok for them to publish the url.
<lifeless> jcsackett: if not, then we should be pretty bold about helping them know, I think.
<lifeless> I have to pop out - big medical day for lynne, I will be back in some hours :(
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 240 - 0:[#######=]:256
<LPCIBot> Project db-devel build #774: STILL FAILING in 6 hr 6 min: https://lpci.wedontsleep.org/job/db-devel/774/
<jelmer__> wgrant: hi
<jelmer__> wgrant: Did you see my email regarding QA?
<poolie> hi all
<wgrant> jelmer__: I replied to the bug.
<sinzui> StevenK, mumble?
<jelmer__> wgrant: thanks
<jelmer__> I guess I missed that.
<sinzui> wallyworld, https://code.launchpad.net/~sinzui/launchpad/person-picker-expand-0/+merge/70082
<wgrant> jelmer__: I saw the bugmail before I saw the direct one (because bugmail still goes to my personal account), so it wouldn't have gone to you directly.
<StevenK> http://pastebin.ubuntu.com/656724/
<wgrant> jelmer__: How confident are you that you will be able to QA this in the next 12 or so hours?
<wgrant> jelmer__: It is getting pretty bad.
<jelmer> wgrant: That should be possible, I can do it first thing tomorrow morning.
<wgrant> Thanks.
<StevenK> https://code.launchpad.net/~stevenk/launchpad/pillarname-vocab/+merge/70091
<LPCIBot> Project devel build #936: STILL FAILING in 5 hr 39 min: https://lpci.wedontsleep.org/job/devel/936/
#launchpad-dev 2011-08-02
<lifeless> bac
<lifeless> *back*
<lifeless> (sorry bac)
<StevenK> lifeless: Haha, I did wonder if you were at your keys or wanted Brad's attention
* lifeless changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 240 - 0:[#######=]:256
* wgrant changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: wgrant | Critical bugs: 241 - 0:[#######=]:256
<wgrant> Anyone around to review https://code.launchpad.net/~wgrant/launchpad/iseriesbugtarget/+merge/69951?
<wgrant> lifeless: You don't have a good suggestion for a primary bug target?
<wgrant> ie. product/distro/DSP?
<lifeless> wgrant: 'thing', 'target', 'context', shrug.
<lifeless> wgrant: I think we want to remove the series special casing eventually, so I don't really care about the name.
<wgrant> True.
<wallyworld> lifeless: following on from last week, could you please look at this mp (and +1 if you are happy). then i will land it for the contributor https://code.launchpad.net/~xaav/loggerhead/export-tarball/+merge/66408
<wgrant>  * 9967 Exceptions
<wgrant> Hi codehosting.
<lifeless> \o/
<poolie> hi wallyworld
<wallyworld> poolie: hi
<poolie> thanks for pushing it
<wallyworld> np
<wallyworld> i got caught up yesterday
<jtv> StevenK, wgrant: I uploaded a package to -partner (on dogfood) and ran publish-ftpmaster.  The pool now contains the .dsc and 2Ãtarball, and the package is in the Sources list, but it's not showing up in the Packages files and I don't see a .deb.  Is that bad?
<StevenK> Has it built?
<StevenK> And 2*tarball?
<wgrant> Needs build + potential acceptance
<jtv> StevenK: orig.tar.gz and debian.tar.gz
<jtv> How do I find out about build and potential acceptance?
<StevenK> What does dogfood.l.n/builders say?
<jtv> 2 builders: ferraz & rubidium, both idle
<jtv> Haven't found my package ("bye") in the logs yet, but the package I based it on ("hello") failed to build.
<jtv> chroot problem.
<jtv> Ah, "bye" built successfully on ferraz.
<wgrant> It probably ended up in NEW.
<wgrant> Yep, there it is.
<wgrant> https://dogfood.launchpad.net/ubuntu/oneiric/+queue
<jtv> The source package page also fails to show a deb.
<wgrant> It doesn't really exist yet.
<jtv> https://dogfood.launchpad.net/ubuntu/+source/bye/1.0
<jtv> Soâ¦ do I need to approve the binary upload like I did the source upload?
<wgrant> Yes.
<jtv> On the way.
<jtv> There we go.
<jtv> And nowâ¦ process-accepted?
<wgrant> publish-ftpmaster should do that, right?
<jtv> You have me there.  Checking.
<jtv> Yup, it does.
<jtv> So then I can just re-run publish-ftpmaster?
<wgrant> Correct.
<jtv> It's mentioning the package now: "(Arch Specific)"
<jtv> I don't know who Arch Specific is, but he seems to be doing something to my package.
<wgrant> "arch specific" means "not architecture independent"
<jtv> wgrant: to be really, really honest I did sort of gather it would be something like that.  :)
<jtv> Anyway, it's done now.  Which seems suspiciously quick.
<jtv> Then again, there won't have been any changes outside of -partner.
<wgrant> And partner is both NMAF and small.
<jtv> NMAF?
<wgrant> No More Apt-Ftparchive
<jtv> Ah, you mean it uses Launchpad's indexing code?
<jtv> Well, all green lights so far.
<wgrant> Right.
<jtv> My package is in Packages for i386, and the +source page shows the i386 build.
<wgrant> Excellent.
<wgrant> And it didn't delete 600GB of archive?
<jtv> Errr
<mtaylor> lifeless: fix all of my problems!
<mtaylor> :)
<jtv> wgrant: the ubuntu-archive dir now contains about 15GB, I think.
<mtaylor> hey all - the new launchpad haproxy stuff is killing me
<jtv> I may lose power.
<wgrant> mtaylor: Oh?
<wgrant> jtv: Sounds good.
<mtaylor> wgrant: it's causing tarmac to have its connections disconnected
<wgrant> mtaylor: It maintains persistent connections?
<mtaylor> wgrant: which is problematic
<mtaylor> wgrant: it ddoes
<wgrant> It probably shouldn't.
<wgrant> It didn't have long-term connections open yesterday, interestingly...
<mtaylor> wgrant: well, according to abently, bzrlib expects to stay connected for some reason
<jtv> wgrant: the biggest chunks being in contents-generationold.  About 2GB in /srv/launchpad.net/ubuntu-archive/ubuntu/pool.
<wgrant> mtaylor: I suspect you want to change your plugin to manage transports a little better.
<wgrant> mtaylor: It can't expect to remain connected for weeks.
<mtaylor> wgrant: it doesn't
<jtv> wgrant: surprisingly, almost the same amount of data in ubuntu-partner.
<mtaylor> wgrant: it just stays connected while it runs tests
<wgrant> Ah.
<wgrant> Someone else is keeping connections open for weeks, so I thought you might be too.
<mtaylor> wgrant: which sometimes is longer than 50s
<mtaylor> nope. that shouldn't be us
<wgrant> So, sounds like your plugin needs to be fixed to not do that.
<wgrant> We can possibly increase the haproxy timeouts for now, if it is causing big problems for you.
<wgrant> But only very temporarily.
<mtaylor> wgrant: rockstar told me to come bug you :) --- and it's actually the core tarmac code
<mtaylor> tarmac holds the connection around the post-merge hook
<wgrant> Hmm.
<wgrant> rockstar: Any reason Tarmac can't handle transports a bit more sensibly?
<mtaylor> wgrant: https://jenkins.openstack.org/view/Nova/job/nova-tarmac/109487/console
<mtaylor> or more excitingly: https://jenkins.openstack.org/view/Nova/job/nova-tarmac/109467/console
<wgrant> That's rather unpleasant.
<lifeless> mtaylor: Use the source monty!
<mtaylor> lifeless: these are not the droids you are looking for
<lifeless> mtaylor: wgrant: so, this is a bzrlib bug.
<wgrant> :( no criticals in that rollout
<lifeless> not the holding open
<lifeless> the handling of the disconnects.
<lifeless> transports are defined stateless.
<lifeless> so are RPC calls.
<mtaylor> lifeless: yay! nothing better than a bzrlib bug!
<lifeless> so the bug is 'when an idle ssh transport is disconnected, bzrlib should not error'
<lifeless> or
<lifeless> 'when an idle ssh transport is interrupted, bzrlib errors'
<lifeless> the haproxy stuff is showing this up very visibly, but the bug pre-existed - bzr didn't seen keepalives, and if ssh wasn't configured to do so it could naturally happen anyhow
<lifeless> however, lets up it for now to 10 minutes.
 * mtaylor filing bzr bug
<mtaylor> bug 819604
<_mup_> Bug #819604: when an idle ssh transport is interrupted, bzrlib errors <Bazaar:New> < https://launchpad.net/bugs/819604 >
<mtaylor> lifeless: you gonna be at the next uds?
<lifeless> no
<lifeless> new sprog (we hope)
<mtaylor> what's a sprog?
<lifeless> child
<mtaylor> OH!
<mtaylor> well, in that case, coming to UDS would be the wrong choice
<mtaylor> (and hopefully congrats)
<lifeless> yes, we think so
<mtaylor> LCA then?
<lifeless> doubtful
<mtaylor> gah. I'm starting to think you're avoiding hanging out with me :)
<lifeless> thats it, totally.
<lifeless> if you wanted to swing by christchurch after UDS we could put you up for a few days
<lifeless> bah
<lifeless> LCA
<mtaylor> ooh. now _that's_ not a bad idea
<mtaylor> and yeah - I read LCA there
<mtaylor> I still haven't been to the south island
<mtaylor> thinking about hanging out with Stewart for a while pre-LCA
<mtaylor> speaking of - I REALLY need to submit talk proposal...
<lifeless> UDS +1 I expect I will be at
<mtaylor> yay!
 * mtaylor lists UDS's and LCA as the important things he should attend
<nigelb> So, recently I used vagrant.
<nigelb> I'm wondering how easy it would be to create a vagrant file to set up LP environement.
<jtv> wgrant: I think partner publication works.  Next up is expedited -security publishing.  For that I need to copy a package into security, right?
<rockstar> wgrant, the reason is because bzr doesn't handle transports more sensibly?
<rockstar> I keep hearing launchpad people pushing back on this, and I understand why, but tarmac pushes as much as it can off to bzr.
<rockstar> I'm unsure why we're timing out so quickly now.
 * rockstar sees that lifeless has already covered this, and goes to bed
<wgrant> jtv: Right.
<wgrant> jtv: You can copy from a public PPA, if you want.
<jtv> Thanks.
<wgrant> Edit before sending? [y/n]: yConnection to bazaar.launchpad.net closed by remote host.
<wgrant> Heh
<StevenK> wgrant: Ouch
<lifeless> StevenK: hi
<lifeless> StevenK: https://code.launchpad.net/~stevenk/launchpad/pillarname-vocab/+merge/70091
<lifeless> StevenK: vocabs depend on stable ordering to paginate
<StevenK> It is stable?
<lifeless> ho ?
<lifeless> how?
<StevenK> It appeared to be when I tested it on DF
<lifeless> its not stable with an order by rank, when two rows can have the same rank
<lifeless> you either need a more granular rank, or a disambiguator
<StevenK> Right, then I want rank first and name second
<StevenK> orderBy='rank- name' ?
<lifeless> for storm? order_by=(SQL('rank'), PillarName.name)
<wgrant> Desc(SQL('rank')), PillarName.name
<lifeless> only if you want it to work
<StevenK> lifeless: It will fail to land anyway, so I'll fix it once I'm sure no other tests have failed
<lifeless> cool
<lifeless> sorry for having to raise this
<StevenK> I should have thought of it, by rights
<lifeless> :)
<LPCIBot> Project devel build #937: STILL FAILING in 5 hr 40 min: https://lpci.wedontsleep.org/job/devel/937/
<lifeless> wgrant: fun stuff for oneiric : https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/819621
<_mup_> Bug #819621: lucid container start failure after calling lxc-stop (fails across reboots) <lxc (Ubuntu):New> < https://launchpad.net/bugs/819621 >
<wgrant> lifeless: I noticed that too, but decided it was probably the avahi or similar that I had installed.
<wgrant> Upsetting to hear that it happens even when the container is not terribly mangled.
<wgrant> (avahi sets odd rlimits for itself, which mean more than one instance can't run on a single kernel without a bit of work)
<wgrant> nproc == 2, what could possibly go wrong.
<lifeless> also having plymouth in the container is just bong
<wgrant> Mmm.
<wgrant> Not necessarily.
<wgrant> It's not just a splash these days.
<wgrant> It's the way upstart communicates with the outside world.
<StevenK> jtv: Are you still playing in dogfood's wading pool?
<StevenK> Bloody *hell* (Error ID: OOPS-2040DOGFOOD18397)
<StevenK> How has DF generated 18,000 OOPSes?
<wgrant> Probably the a-f warnings.
<StevenK> And the publisher is running, so the webapp is being horridly slow.
<wgrant> :D
<StevenK> It can't even load bugs.dogfood.l.n
<wgrant> wallyworld: Hmm, so the question subscription stuff is based on the *old* Bugs stuff?
<wallyworld> no, how did you think that?
<wgrant> "Direct subscribers" and "Also notified"
<wgrant> Old, but more sensible, terms.
<wallyworld> that's how questions subscriptions have always worked. i just made it use ajax
<wallyworld> instead of html forms
<wallyworld> so i extended the new subscribers_list portlet
<wgrant> Right.
<wallyworld> so, you ok with the change?
<jtv> StevenK: still running
<wgrant> wallyworld: Would be nice if we eventually integrated the same self-subscription stuff that Bugs introduced, but small steps.
<wallyworld> wgrant: yes, one thing at a time. it's now much better than it was last week. it's now possible to unsubscribe without SQL :-)
<wgrant> Yup.
 * wallyworld off to soccer
<adeuring> good morning
<allenap> Good morning/evening wgrant :) Got time for a very short review? https://code.launchpad.net/~allenap/launchpad/dsd-base-view-dont-modify-globals-bug-809985/+merge/70076
<wgrant> allenap: that's rather evil, but OK!
<wgrant> Approved.
* wgrant changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 241 - 0:[#######=]:256
<allenap> wgrant: Thanks :)
<jelmer> g'morning
<mrevell> Morning
<allenap> Morning jelmer :)
<allenap> jelmer: I saw that Dulwich made the front page of Hacker News.
<jelmer> allenap: yeah :)
<jelmer> allenap: I've met the Google code folks and have been in contact with them. They've contributed a bunch of patches which should help with performance.
<allenap> jelmer: Also, bug 808035 is at the top of the deploy queue and needs testing :)
<_mup_> Bug #808035: AttributeError: 'NoneType' object has no attribute 'write' during git import <oem-services> <qa-needstesting> <regression> <Launchpad itself:Fix Committed by jelmer> < https://launchpad.net/bugs/808035 >
<jelmer> That should also have a significant impact on git code import performance.
<jelmer> allenap: Yep, it's on the top of my todo list too.
<allenap> jelmer: That's really cool :)
<lifeless> adeuring: hiya
<danilos> mrevell, hi, do you have any time today to help move the docs forward and potentially announce the change to maintainers by email (import queue stuff)?
<mrevell> danilos, Yes, certainly. In 30 minutes.
<danilos> mrevell, excellent, let's talk then
<adeuring> hi lifeless
<lifeless> adeuring: https://code.launchpad.net/~adeuring/launchpad/bug-739052/+merge/69625 - you're not landing it yet ? [I can't see the column issue addressed]
<adeuring> lifeless: now running through ec2
<adeuring> forgot that yesterday...
<lifeless> adeuring: and there were a bunch of suggestions that seem forgotten too :(
<adeuring> I'm also working on the remaining suggestions you made
<lifeless> adeuring: oh cool! - are you thinking of two separate landings ?
<adeuring> lifeless: actually, three, to keep the diffs to review small
<lifeless> sure thing; I'd be delighted to review those for you
<lifeless> adeuring: have I mentioned how glad I am that you're doing this.
<adeuring> lifeless: too late for this one : https://code.launchpad.net/~adeuring/launchpad/bug-739052-2/+merge/70044 ;)
<adeuring> lifeless: and thanks :)
<adeuring> (sorry, I have some reaally long lag between typing IRC messages and looking at incoimng...)
<lifeless> heh no worries
<lifeless> adeuring: so why won't (col1 >= M1 and col2 >= M2 and colN >MN) work ?
<lifeless> oh, I see
<lifeless> adeuring: however, using a tuple will work, I think ?
<lifeless> hmmm, only if the direction on all the columns is consistent
<adeuring> lifeless: no, I think we need, for three sort columns:
<adeuring> col1 == memo1, col2 == memo2, col3 > memo3, then...
<adeuring> col1 == memo1, col2 > memo2
<adeuring> then col1 > memo1
<lifeless> adeuring: what about
<lifeless> (col1, col2, col3) >= (memo1, memo2, memo3)
<adeuring> that wouldn't change much
<adeuring> problem is:
<adeuring> when the value in col1 increases, the values in col2, col3... starting from the minimum
<adeuring> so we end up with N conditions for N sort columns
<adeuring> lifeless: I think we have at least three options to "join" these N conditions: a more or less huge OR (implemented for mow), or a UNION ALL on sub-selects for each conditions
<adeuring> or
<adeuring> we can iterate in Python over the conditions
<lifeless> adeuring: if the direction is consistent, a tuple works
<adeuring> lifeless: ah, right!
<lifeless> http://pastebin.com/mcShr1D2
<adeuring> lifeless: but how do we express this in SQL?
<lifeless> just so
<adeuring> lifeless: cool!
<adeuring> so, the content of my next branch is obvious ;)
<lifeless> :)
<lifeless> we may have a lot where we can't do this
<lifeless> but we can probably group all the adjacent same-direction columns anyhow
<adeuring> lifeless: right
<henninge> jml: Hi! ;)
<jml> henninge: hi
<henninge> jml: I got a failed ec2 run for your branch.
<jml> henninge: me too, but no actual failures
<henninge> jml: right
<henninge> it's the "smtp message exeeds size limit"
<henninge> jml: same for you?
<jml> henninge: I've corrected the default value as per StevenK's comments. I suspect that would have caused some tests to fail.
<jml> henninge: yes.
<lifeless> that smtp error means a catastrophic failure
<henninge> lifeless: you mean "too many failures, the mail gets too big" ?
<lifeless> yes
<henninge> jml: I see you pushed, shall I start the landing again?
<jml> henninge: yes. thank you.
<henninge> ok
<lifeless> allenap: there is a feature flag you can use for profiling too
<lifeless> allenap: the default profile dir is $cwd, I think.
<allenap> lifeless: Oh neat. I was just shuffling my wiki notes; I think that one is from back in the days when BjornT_ ran the bugs team :) Not sure if it'll even work any more.
<jtv> Any reviewers in the house?  â https://code.launchpad.net/~jtv/launchpad/bug-819674/+merge/70135
<jml> who actually uses checkUpload via the API?
<LPCIBot> Project devel build #938: STILL FAILING in 5 hr 44 min: https://lpci.wedontsleep.org/job/devel/938/
<jelmer> jml: I would imagine things like request-sync
<jelmer> jml: to do things like determine if the current user requires sponsorship
<jml> jelmer: ah, yeah, that would be a thing.
<jml> for checking PPA uploads, it's not particularly useful
<jml> since it errors when it doesn't recognize the source package, and insists that you supply component & pocket
<jelmer> yeah, it's a bit odd
<lifeless> night all
<jelmer> g'night lifeless
<jelmer> jtv-afk: trade you for https://code.launchpad.net/~jelmer/launchpad/logginguifactory-fixes/+merge/70144 ?
<StevenK> The publisher takes 4+ hours on DF? Srsly?
<bigjools> StevenK: it sometimes does
<abentley> adeuring: Is it considered a feature that https://bugs.launchpad.net/mahara/+bug/630891%20/+index works?
<_mup_> Bug #630891: Unable to delete Notification messages <Mahara:Fix Released by richard-mansfield> < https://launchpad.net/bugs/630891 >
<abentley> adeuring: i.e. a URL with whitespace in it?
<adeuring> abentley: I have no idea...
<abentley> adeuring: It seems bizarre, and it means that https://bugs.launchpad.net/mahara/+bug/630891%C2%A0/+index fails due to encoding issues.
<_mup_> Bug #630891: Unable to delete Notification messages <Mahara:Fix Released by richard-mansfield> < https://launchpad.net/bugs/630891 >
<adeuring> abentley: I don't see how these two URL-oddities are related
<abentley> adeuring: If URLs with spurious characters were not permitted, the second would give a LocationError instead of trying, and failing to render.
<abentley> adeuring: i.e. a "Not found"/ "Lost something"?
<adeuring> abentley: sure, a "not found" would be more reasonable, but the failure for your second URL is a separate issue: A unidoce decoding problem
<adeuring> abentley: we would get that too for a project name like "mÃ¼lltÃ¼te"
<adeuring> but such names a not allowed
<abentley> adeuring: Our URLs don't have non-ascii characters, so they can't have a unicode decoding problem.
<abentley> adeuring: Except bug URLs, which can have spurious non-ascii characters.
<adeuring> abentley: do we have a referer for the \xa0 / %C2%A0 URL?
<adeuring> abentley: I mean: you can always type URLs manually
<abentley> adeuring: No, but it was googlebot.
<adeuring> abentley: so the URL has been somewhere/somehow generated. So we have two issues: the unicode error, and the fact that a spurious space at least in bug URLs does not result in a 404
<abentley> adeuring: Yes, that's my view.  It hardly seems worth fixing the encoding bug if non-ascii URLs are supposed to be verboten.
<abentley> adeuring: Who knows what other bugs non-ascii URLs might reveal?
<adeuring> abentley: yeah,,,
<adeuring> abentley: but regarding the working URL with the %20:
<adeuring> Python 2.7.1+ (r271:86832, Apr 11 2011, 18:13:53)
<adeuring> [GCC 4.5.2] on linux2
<adeuring> Type "help", "copyright", "credits" or "license" for more information.
<adeuring> >>> int('1')
<adeuring> 1
<adeuring> >>> int('1 ')
<adeuring> 1
<adeuring> abentley: I guess that the travesal code does just such a simple int('630891 ')
<abentley> adeuring: Interestingly: ValueError: invalid literal for int() with base 10: '2\xa0'
<adeuring> abentley: try int(u'1\xa0')
<abentley> adeuring: Fascinating.  So that is only permitted for unicode strings, because only in unicode strings does \xa0 refer to the same codepoint consistently.
<adeuring> abentley: doesn't the "can't encode character u'\xa0'" in the traceback indicate that dict['PATH_INFO'] already _is_ a unicode object?
<abentley> adeuring: yes, that's what I think.
<adeuring> abentley: so again not a "clear border" where 8-bit sequences with some encoding are supposed to live on one side and unicode objects on the other...
<abentley> adeuring: Right.  Requests appear to use a "sane_environment" where values are unicode, but converting the browser request to a web service request appears to re-convert the environment.
<adeuring> abentley: so.... a trvial but bad workaround would be to convert to utf-8 in this case.
<abentley> adeuring: Might be a necessary workaround, depending on how these calls are structured.
<adeuring> abentley: if you want more fun, try this URL: https://bugs.launchpad.net/mahara/+bug/630891%C3%A4/+index
<_mup_> Bug #630891: Unable to delete Notification messages <Mahara:Fix Released by richard-mansfield> < https://launchpad.net/bugs/630891 >
<adeuring> quite different error...
<adeuring> %C3%A4 is a 'Ã¤'
<abentley> adeuring: Yeah, that's kind of error I'd expect.
<adeuring> abentley: I assume that this error is raised for a failed int('630891Ã¤')
<adeuring> (for the bug id)
<abentley> adeuring: me too.
<abentley> adeuring: a0 is non-breaking whitespace.
<adeuring> right
<adeuring> so it might be a replacement from the regular '\x20' space
<abentley> adeuring: Maybe.  Since googlebot is the source, I imagine it's in some web page, though.
<adeuring> abentley: right. And the s/\x20/\xa0/ might be deliberate: To avoid wrapping the URL...
<adeuring> though as far as I know most webbrosers ignore the "don't wrap" for \xa0
<abentley> adeuring: but of course URLs shouldn't have whitespace.
<adeuring> abentley: sure, but we can't stop the world mangling our URLs ;)
<abentley> adeuring: we can, it's just hard to get venture capital :-)
<adeuring> abentley: it might help to add a check like "strip(path_element) == path_element" before the int(path_element)
<adeuring> ...where path element is the bug ID
<abentley> adeuring: Yeah, that would work, or re.match('[0-9]*', path_element)
<adeuring> abentley: right
<adeuring> a nitpick: re.match('^[0-9]+$')
<adeuring> abentley: for even more fun: https://bugs.launchpad.net/mahara/+bug/630891%E4/+index
<_mup_> Bug #630891: Unable to delete Notification messages <Mahara:Fix Released by richard-mansfield> < https://launchpad.net/bugs/630891 >
<abentley> adeuring: what the??
<adeuring> abentley:  %E4 is not vaild UTF8
<adeuring> cool, right?
<abentley> adeuring: That's wacky.
<abentley> adeuring: merge proposals are also affected.
<adeuring> abentley: interesting. Again via an integer ID?
<abentley> adeuring: Right.
<abentley> adeuring: I'm gonna guess that everything that has an integer ID is affected.
<adeuring> abentley: seems this is worth a general helper function "check an int ID in a URL wfor all sorts of oddities"
<abentley> adeuring: Yes, though I expect this kind of thing will rot.  Maybe we should just reject URLs with whitespace from the beginning.
<abentley> adeuring: Or more precisely, URLs with whitespace in the PATH.
<adeuring> abentley: the you can also add checks for the other issues: any byte value outside the ASCII set
<adeuring> more precisely: all bytes in the URL should have integer values between 33 and 126
<adeuring> ...127
<adeuring> ...after replacing the %xx encoding
<adeuring> ouch.. the latter check should _not_ be done for the part after '?'
<abentley> adeuring: Right, only the PATH.
<adeuring> yep
<LPCIBot> Project db-devel build #775: STILL FAILING in 5 hr 59 min: https://lpci.wedontsleep.org/job/db-devel/775/
<LPCIBot> Project devel build #939: STILL FAILING in 5 hr 45 min: https://lpci.wedontsleep.org/job/devel/939/
<jcsackett> sinzui: are you free to mumble?
<sinzui> jcsackett, I am otp. I will be available in 45 minutes
<jcsackett> sinzui: dig.
<sinzui> jcsackett, I can talk now
<jcsackett> sinzui: cool.
<benji> Anyone want a small (and possibly even slightly fun) JS review? https://code.launchpad.net/~benji/launchpad/bug-809511/+merge/70219
<benji> sorry, gary claimed it so you all missed out
<allenap> Hi, can anyone take care of reverting rev 13574 (from bug 810290). It's qa-bad, but I really have to go.
<_mup_> Bug #810290: no way to set scopes for incoming mail <feature-flags> <mail> <qa-bad> <Launchpad itself:Fix Committed by mbp> < https://launchpad.net/bugs/810290 >
<lifeless> 13574 ?
<lifeless> allenap: does it work *without* the feature rule in place ?
<LPCIBot> Project db-devel build #776: STILL FAILING in 5 hr 39 min: https://lpci.wedontsleep.org/job/db-devel/776/
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #940: FIXED in 5 hr 48 min: https://lpci.wedontsleep.org/job/devel/940/
<sinzui> StevenK, mumble?
<wallyworld> sinzui: my sound is @!%@!%$!@^. rebooting
<lifeless> matsubara-afk: wow, nice defect finding
* StevenK changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: StevenK | Critical bugs: 241 - 0:[#######=]:256
<poolie> hi all
<wgrant> Hi poolie.
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: StevenK | Critical bugs: 248 - 0:[#######=]:256
<wgrant> poolie: Seen the latest on bug #810290?
<_mup_> Bug #810290: no way to set scopes for incoming mail <feature-flags> <mail> <qa-bad> <Launchpad itself:Triaged by mbp> < https://launchpad.net/bugs/810290 >
<poolie> not yet, was just wondering about it
<poolie> ah
<poolie> sorry
<poolie> so is that ok just rolled back, or is anything else needed?
<wgrant> Should be all OK.
<wgrant> Just takes 8 hours.
<poolie> to roll it back out?
<poolie> :/
<poolie> i did have some worries about whether this was sufficiently tested
<wgrant> Yeah, because it missed buildbot by 3 minutes.
<wgrant> So needs to wait for two runs.
<poolie> i guess if it's trapping there the answer is that this code is not reached at all in the test suite
<sinzui> wallyworld, is this what you were expecting? http://pastebin.ubuntu.com/657556/
 * wallyworld looks
<wallyworld> sinzui: yes, i think that's a bit nicer?
<sinzui> The tests passed
<sinzui> I will commit
<wallyworld> \o/
<wallyworld> thanks
<sinzui> wallyworld, Changes are pushed
<wallyworld> thanks, will approve and organise a mentor
#launchpad-dev 2011-08-03
<wgrant> Thanks StevenK.
<wgrant> You are not yet imprisoned?
<jelmer> What has StevenK done now?
<StevenK> jelmer: On the call today I said I have a rental inspection, so if I'm not around for the rest of the day it's because I've been arrested for killing my landlord.
 * jelmer tries to imagine what a rental inspection is
<poolie> like this: http://www.youtube.com/watch?v=aCiYmCVikjo
<jelmer> heh
<StevenK> jelmer: Where the landlord comes over to inspect the place you're renting to make sure you're taking care of it
<wgrant> StevenK: It looks like the FileBugAPI XML-RPC call has been used around once in the last three months.
<jelmer> StevenK: I don't think my landlord is legally allowed to do that
<StevenK> Haha, around once
<poolie> wgrant: coincidentally i was wondering this morning if there are or should be postmortems for rollbacks
<StevenK> wgrant: Zero is around once?
<StevenK> wgrant: It seems cronscripts/create-debwatches.py uses debsync?
<jelmer> StevenK: while you're here.. can I add a branch to your queue?
<StevenK> jelmer: I just reviewed one of yours
<StevenK> jelmer: Or do you mean a different branch?
<jelmer> StevenK: ah, race condition.. thanks for the review :)
<StevenK> And create-debwatches isn't actually run
<StevenK> Orsum
<wgrant> No, chromium, you may not have 7GB of RAM.
<StevenK> Bwaha
<wgrant> StevenK: Right, I'm pretty sure that create-debwatches and update-debwatches have never been used.
<wgrant> They were written at the end of 2004.
<wgrant> And haven't really been touched since.
<wgrant> StevenK: Heh, look at r7675.771.2
<wgrant> One of mine: "Port create-debwatches.py away from IDS.publishedBinaryPackages(). The script didn't work anyway, so I'm just going to pretend that I haven't broken it more."
<StevenK> Haha
<wgrant> So, IMO you should delete it and update-debwatches.py, and probably trherefore debsync.
<StevenK> steven@liquified:~/launchpad/lp-branches/kill-cruft% bzr di | diffstat | tail -n 1
<StevenK>  7 files changed, 1890 deletions(-)
<wgrant> What's removed?
<StevenK> wgrant: http://pastebin.ubuntu.com/657575/
<wgrant> StevenK: You've confirmed we have a bugzilla exporter?
<wgrant> This script was clearly written for the Ubuntu migration and probably hasn't been touched since, but best to check.
<StevenK> wgrant: scripts/migrate-bugzilla-initialcontacts.py should probably die a firey death too
<wgrant> Probably, yes.
<StevenK> wgrant: You don't mean things like lp.bugs.externalbugtracker.bugzilla?
<wgrant> No.
<wgrant> That's the checkwatches backend.
<wgrant> The bugzilla exporter won't be in our tree.
<StevenK> Ah
<wgrant> Meh, delete it anyway. There probably is one.
<StevenK> Haha
<StevenK> As a serious question, do we actually care?
<wgrant> No.
<lifeless> about what ?
<wgrant> The ancient bugzilla import script.
<lifeless> ubuntu bugzilla to lp?
<lifeless> does it even work now ?
<wgrant> Very probably not.
<wgrant> Hence its imminent demise.
<wgrant> lifeless: The XML-RPC FileBugAPI (https://help.launchpad.net/MaloneXMLRPC) has been used four times this year.
<wgrant> Can we kill it?
<lifeless> we what now?
<wgrant> Hm?
<StevenK> Can haz review?
<poolie> wgrant: how much extra work does a rollback cost?
<poolie> 10m, an hour, a day?
<poolie> i mean across losas and developers
<lifeless> poolie: you might like the mail I just sent
<lifeless> poolie: which puts some structure on this for the pre-deploy case.
<wgrant> poolie: LOSA time: should only be a few minutes, but I don't know how long they spend watching it.
<poolie> ah ok
<lifeless> poolie: things that have been deployed, and then rolledback incur the pre-deploy pipeline overheads *as well*
<poolie> i was just writing up a few thoughts on this change
<wgrant> Well, if we catch it before deployment, no LOSA time at all.
<lifeless> poolie: which change ?
<poolie> mail-scope
<lifeless> poolie: so it never got deployed
<lifeless> poolie: no cost to losa
<poolie> it got deployed to qastaging
<poolie> no?
<poolie> or is that not called 'deployed'?
<lifeless> an unqualified 'deployed' means production only.
<wgrant> That's not "deployed"
<lifeless> poolie: the mail scope change falls directly under the performance tuesday mail I just sent.
<wgrant> Blah.
<poolie> what is that called then, "deployed to qastaging"?
<wgrant> Or just "on qastaging"
<wgrant> wallyworld: The Subscribe spinner on your Answers work is a bit odd.
<wallyworld> i reused what was there already
<wallyworld> we can iterate if required
<wgrant> It doesn't appear until a few seconds after the click :(
<wallyworld> i call the method to display the spinner inthe onclick
<wgrant> Huh.
<wallyworld> well i thought i did
<wallyworld> i will have to check
<poolie> lifeless: yeah it totally does
<wgrant> It seems to make an XHR request, show the spinner, then make another.
<wgrant> lifeless: "20 minutes DEPLOY -> deployment -> fails or"
<wgrant> Is that the 75 minutes or so to qastaging?
<lifeless> wgrant: has it degraded?
<poolie> i am trying to develop more sensitivity to risky landings by looking back at things that failed
<lifeless> wgrant: but yeah, thats what I was refering to.
<poolie> or, that could have been risky but worked
<lifeless> poolie: I think that we have a skewed environment because of two things:
<lifeless>  - long pipeline
<lifeless>  - single pipeline
<poolie> yes, that's clearly true
<lifeless> as we get more services with independent pipelines, and a faster pipeline
<lifeless> then we can iterate more in the direction responsive companies are talking about
<wallyworld> wgrant: looks like you are right. it was supposed to be displayed before the first xhr call, but instead happens after. i'll raise a bug and fix. thanks
<wgrant> lifeless: It only checks every half hour, and takes nearly an hour.
<lifeless> I don't think applying their basic conclusions to us today will work at all well, though knowing their analysis methods / model for the problem is useful.
<wgrant> wallyworld: Thanks! If possible, it mihgt be nice to also have it change the "(+) Subscribe" into "(o) Subscribing" like Bugs used to do, and hopefully will again soon.
<lifeless> wgrant: please send an errata
<lifeless> wgrant: I would appreciate that
<wallyworld> wgrant: i thought about that approach but initially just wanted to reuse as much as possble what was there already
<wallyworld> lifeless: might you have time to +1 that loggerhead mp?
<wallyworld> https://code.launchpad.net/~xaav/loggerhead/export-tarball/+merge/66408
<lifeless> wallyworld: I haven't re-read it yet
<wallyworld> np
<lifeless> wallyworld: this week has been -very- bitsy. Its a high pri for me.
<wallyworld> that's cool. sorry for nagging :-)
<wgrant> wallyworld: Good plan.
<lifeless> nothing to apologise for
<lifeless> I'm holding up someones contribution, which sucks.
<poolie> lifeless: sent; would appreciate your thoughts on it some time
<StevenK> lifeless: You were looking over my kill-cruft MP to review it, or out of interest?
 * wallyworld puts on earplugs - electrician installing new solar inverter - *very noisy*
<lifeless> StevenK: s/interest/horror/
<lifeless> StevenK: :)
<lifeless> StevenK: I think you should JFDI
<poolie> oh no, the inverter's upside down!
<lifeless> poolie: I wanted to touch base with you anyhow; would you like a chat now ?
<poolie> sure
 * wallyworld slaps forehead at poolie's bad pun
<poolie> pots?
<lifeless> I'm on the laptop, so can do sip or skype or whatever
<poolie> sip
<poolie> 'currently unavailable'
<wgrant> lifeless: So, I think we should just drop the XML-RPC method. It's been pretty much deprecated since the API was introduced, seems to be used once every couple of months, and whoever is using it can spend the 5 minutes rewriting their script, if indeed someone is actually using it.
<StevenK> TBH, I agree with lifeless. Let's FF it first
<StevenK> And soyuz-upload.txt is broken
<StevenK> Can we delete that yet?
<wgrant> Wha?
<wgrant> It passed ec2..
<StevenK> http://pastebin.ubuntu.com/657616/
<wgrant> Yeah.
<wgrant> Maybe the race was real.
* jtv changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: StevenK, jtv | Critical bugs: 248 - 0:[#######=]:256
<wgrant> StevenK: I think I will just rip the poppy section out of soyuz-upload.txt
<wgrant> Since test_poppy.py tests it all already.
<StevenK> Excellent
<wgrant> In fact, this removes calls to poppy from outside lp.poppy.
<wgrant> Handy.
<StevenK> \o/
<StevenK> Do you need a review?
<wgrant> It's only test changes, so I will self-review it.
<jtv> hi wallyworld â just letting you know I'm on the mentoring request.
<wallyworld> jtv: thanks!
<jtv> I've been doing disappointingly little of that lately, so high time I caught up.
<lifeless> wgrant: in your lxc's, have you been configuring up postfix ?
<lifeless> wgrant: or something like nullmailer ?
<wallyworld> wgrant: the subscription portlet's existing behaviour was to display progress  after the initial xhr call. i can't fix that for "subscribe someone else" or for when "subscribe me" displays an entry in the list but can fix it for answers as it is currently used
<lifeless> I wonder if 122 /    3  Person:EntryResource:searchTasks is a regression
<wallyworld> StevenK: want a very small review? https://code.launchpad.net/~wallyworld/launchpad/subscriber-portlet-spinner/+merge/70247
<StevenK> wallyworld: r=me, but I'm not very comfortable with my JS-foo
<wallyworld> StevenK: thanks. the change was just moving a few lines around, so should be ok
<lifeless> erm
<lifeless> https://launchpad.net/ubuntu/+source/apt/+index
<lifeless> click on the expanders
<lifeless> they all seem to go 'failed to fetch package details'
<lifeless> and retry reloads the whole page
<lifeless> is this known ?
<wgrant> lifeless: I haven't.
<wgrant> lifeless: Just whatever rocketfuel-setup installs, which is IIRC nothing.
<lifeless> wgrant: recommends are on by default
<lifeless> wgrant: so it pulls in postfix
<wgrant> Ah, no, I disable recommends.
<wgrant> Halves the downloads or so.
<lifeless> wgrant: right; see my mail to -dev :P
<wgrant> I always add -o Apt::Install-Recommends=no, mostly for speed and clutter reasons.
<wgrant> But have never had the heart to do it in devel.
 * wgrant reads.
<wgrant> Hah
<wgrant> Erm.
<wgrant> Isn't that top OOPS meant to be fixed?
<wgrant> But maybe the fix wasn't a rollback and the original rev wasn't marked qa-bad :(
<lifeless>  145.73s  OOPS-2040CCW4   http://www.urbanvelo.org/
<lifeless>    ?
<wgrant>  943 LocationError: (<zope.browserpage.metaconfigure.SimpleViewClass from /srv/launchpad.net/production/launchpad-rev-13552/lib/lp/registry/browser/../templates/object-timeline-graph.pt object at INSTANCE-ID>, 'getCacheJSON')
<lifeless> I suspect not a rollback, not marked bad.
<lifeless> In my mail about pipelines I've covered this sort of change FWIW
<lifeless> I'd rather we never do such things.
<lifeless>  https://launchpad.net/ubuntu/+source/apt/+sourcepub/1635507/+listing-archive-extra
<lifeless> is 404ing
<lifeless> (from https://launchpad.net/ubuntu/+source/apt/+index)
<lifeless> wgrant: right any bells? I think you refactored stuff in this area?
<wgrant> Possibly the expander refactoring a couple of weeks ago.
<wgrant> I think that should be /ubuntu/+archive/primary, not /ubuntu/+source/apt.
<wgrant>     /**
<wgrant>      * If a page wants to use this outside of an archive context then it
<wgrant>      * can define LP.cache['archive_context'], which should be a
<wgrant>      * full or relative URL to the context archive page required.
<wgrant>      */
<wgrant> Suspicious
<wgrant> And indeed, that's set correctly on DSP:+index...
<wgrant> So why is it ignored.
<lifeless> heh, you've got the bit between teeth huh.
<lifeless> should I file a bug for this, or do you want to ?
<wgrant> Bit between teeth?
<lifeless> http://www.usingenglish.com/reference/idioms/bit+between+your+teeth.html
<wgrant> Ah
<StevenK> Build Source Stamp: [branch bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel] 13589
<StevenK> WGRANT!
<lifeless> if a horse grabs onto the bit, the bridle can no longer control the angle of its head, because the teeth can take a -lot- of pressure vs the back of the jaw
<StevenK> Blamelist: William Grant <william.grant@canonical.com>
<wgrant> StevenK: The testfix landed hours ago!
<StevenK> Ah, it's soyuz-upload again
<wgrant> Again?
<wgrant> This is the build you complained about while it was in progress!
<StevenK> That was ec2
<StevenK> That failure was from buildbot
<wgrant> Oh.
<wgrant> I presumed you saw the failure on buildbot.
<StevenK> Just then, from it's bleating mail
<StevenK> wgrant: You've killed poppy ftp, can we nail https://bugs.launchpad.net/launchpad/+bug/29645 shut?
<_mup_> Bug #29645: Weird requirement to poll the ftp.sock during the upload session <lp-soyuz> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/29645 >
<wgrant> StevenK: I plan to go through all poppyish bugs soon, yeah.
<StevenK> Cool
<wgrant> lifeless: Ah!
<wgrant> lifeless: It's a fresh regression.
<wgrant> Aaron's thing.
<wgrant> It now clobbers LP.cache instead of just adding new items to it.
<lifeless> wgrant: urgh
<wgrant> Checking to see what else is affected...
<lifeless> wgrant: I'll let you file it
<lifeless> I guess we need to revert prod ?
<wgrant> Mmm. QA-wise we're in an OK state.
<lifeless> we don't get js exceptions to count oopses
<wgrant> We should be able to deploy this fix in 10 hours.
<wgrant> So, maybe.
<wgrant> I guess we should.
<lifeless> so the fallout is unknown
<wgrant> We have to go back all the way, though.
<lifeless> yes
<wgrant> It was my deployment, not the US one :(
<lifeless> do you happen to know the old good evno ?
<wgrant> The one before 13548... which was 13543
<wgrant> This was reported 22 hours ago, but I didn't notice it :/
<wgrant> lifeless: This should be the only fallout of this type.
<wgrant> There are a couple of other templates which manipulate LP.cache, but they do it on domready and are working fine.
<lifeless> wgrant: how was it reported ?
<wgrant> 16:58:00 < diwic> "Failed to fetch package details. Retry" <- is this a known launchpad bug?
<wgrant> In #launchpad.
<lifeless> ah, didn't miss a bug report.
<lifeless> thanks
<wgrant> Bug #820174, anyway
<_mup_> Bug #820174: Expanders on DistributionSourcePackage:+index broken by LP.cache changes <regression> <Launchpad itself:In Progress by wgrant> < https://launchpad.net/bugs/820174 >
<wgrant> StevenK: Just failed for me in ec2 too :(
<wgrant> Yet I have an email here, "Test results: murder-poppy => devel: SUCCESS" :(
<StevenK> Even with your change in devel?
<wgrant> No.
<wgrant> I know that's not racy :)
<StevenK> Strange
<wgrant> Oh?
<jtv> wallyworld: I'm afraid I reverted to my usual nit-picky self on that branch.
<StevenK> wgrant: That it failed, but you get a success message
<wgrant> Ah, no, you misunderstand.
<wgrant> The branch passed ec2. The test did not fail.
<wgrant> It landed.
<wgrant> Then broke buildbot and every subsequent ec2 run.
<jtv> StevenK: are you ready to exact sweet, sweet revenge for all the horrible things I've done to you?
<StevenK> Heh
<wallyworld> jtv: i'm looking now, thanks
<StevenK> jtv: No, because I'm not yet in the same country as you ...
<jtv> StevenK: you clearly have not learned to appreciate the full potential of the internet.
<StevenK> I can't kick people in the face over the Internet.
<wgrant> lifeless: Would you prefer a full revert, or https://code.launchpad.net/~wgrant/launchpad/fix-dsp-index/+merge/70249?
<wgrant> Since it can skip ec2.
<lifeless> wtf
<lifeless>  robertc@lucid-test-lp:~/source/launchpad/lp-branches/working$ ls /etc/postgresql/
<lifeless> 8.2
<wgrant> Erm.
<StevenK> Fail
<StevenK> wgrant: That MP looks good to me
<lifeless> wgrant: is the other template thing fixed too already ?
<wgrant> Wasn't 8.2 removed just after hardy?
<lifeless> wgrant: no freaking idea
<wgrant> lifeless: Which?
<wgrant> The top OOPS from yesterday?
<lifeless>  lsb_release -a
<lifeless> No LSB modules are available.
<lifeless> Distributor ID: Ubuntu
<lifeless> Description:    Ubuntu 10.04 LTS
<lifeless> wgrant: yea
<StevenK> postgresql-8.2 |    8.2.7-1 | hardy/universe | source, amd64, i386
<jtv> StevenK: I still say you're not appreciating (1) the full potential of the internet or (2) the awesome violent scale of my physical reaction to face-kicking attempts.  Hence my offer of safe retribution across the internet.
<wgrant> lifeless: Was fixed in r13560
<wgrant> lifeless: So the fix is on prod already.
<wallyworld> jtv: you certainly are very detailed. i'll check in with curtis tomorrow. thanks again.
<StevenK> wallyworld: s/details/pedantic/
<wallyworld> lol
<StevenK> jtv: How?
<jtv> wallyworld: it's the risk of over-escaping date formatting that has me worried.
<lifeless> wgrant: ok, I'm fine with rollforward if we're confident of no other sites.
<jtv> StevenK: by reviewing one of my branches!  https://code.launchpad.net/~jtv/launchpad/bug-819674/+merge/70135
<StevenK> Ah, that branch.
<lifeless> wgrant: we will want the original bug to be bad-commit flagged and this new one flagged rollback too though.
<lifeless> wgrant: I think.
<wgrant> lifeless: Yup.
<wallyworld> jtv: yes, i think that's a fair question to ask
<StevenK> I am not comfortable doing so, since I don't like the idea of the publisher being SIGINT'd
<wallyworld> i suspect it won't be an issue but still.....
<wgrant> StevenK/lifeless: Can I have a +1 on the MP?
<StevenK> wgrant: r=me
<wgrant> Thankyou sir.
<jtv> StevenK: fair enough.  It all ends up being nice & safe though.
<StevenK> I don't trust it either
<StevenK> How can you be *certain* that the publisher is safe to Ctrl-C
<StevenK> It's a can of worms
<wgrant> As long as the backup switch happens only around publish-distro and not process-accepted, it's not too bad,.
<wgrant> The worst it can do is delay.
<StevenK> Bleh, and my kill-cruft branch has failed due to soyuz-uploads
<wgrant> :(
<StevenK> RETRIBUTION
<jtv> Feels good, doesn't it?
<jtv> Anyway, if wgrant says it's not too bad, it's probably not too bad.  :)  I mapped out and simplified the whole directory dance when I re-did publish-ftpmaster, and now nothing risky gets done in the "real" dists directory until all the hard work is done.
<StevenK> jtv: I was talking about retribution for wgrant
<jtv> Oh.
<wgrant> Just don't revert whatever process-accepted did, and you won't die.
<wgrant> Hmm.
<wgrant> So, confirmed, nothing else touches LP.cache in bad ways.
<wgrant> The rest of that rev may be good.
<StevenK> Then land your fix
<StevenK> IMO
<wgrant> It landed a while ago.
<StevenK> Then speed up buildbot
 * wgrant points at lifeless and LXC.
<lifeless> looks like I'm ready to try parallel testing again, having beaten oneiric lxc into submission
<jtv> ! \o/
<wgrant> lifeless: I see the restart problem has been diagnosed.
<lifeless> wgrant: there are some terrible hacks under the hood, something about O makes them more visible.
<jtv> wgrant: would you be able to review my branch perhaps, since StevenK is too chicken?
<wgrant> Maybe.
<jtv> Maybe thanks.
<wgrant> jtv-eat: So this is just improving tests?
<wgrant> There seems to be no functional change.
<lifeless> wheee
<lifeless> localpackagediffs fail - generated two OOPSes :)
<lifeless> https://launchpad.net/ubuntu/oneiric/+localpackagediffs?field.name_filter=apt&field.package_type=ignored&field.package_type-empty-marker=1
<lifeless> and hit refresh
<lifeless> Module lp.services.propertycache, line 116, in __get__
<lifeless> value = self.populate(instance)
<lifeless> Module lp.registry.browser.distroseries, line 1081, in cached_differences
<lifeless> status = package_type_dsd_status[self.specified_package_type]
<lifeless> KeyError: u'ignored'<br />
<wgrant> Hmmm.
<lifeless> (apt is currently generating a diff)
<wgrant> That page does have a thread-safety issue, but it's behind a flag and shouldn't cause this.
<wgrant> It seems to only affect some appservers.
<wgrant> It renders every so often.
<LPCIBot> Project devel build #941: FAILURE in 6 hr 8 min: https://lpci.wedontsleep.org/job/devel/941/
<StevenK> Bah, there is an import violation that requires a storm fix
<wgrant> Excellent.
<wgrant> All remaining QA is European.
<StevenK> allenap and danilos, right?
<wgrant> And rvba.
<jtv> wgrant: yup, I just didn't realize it until I'd written the tests.
<wgrant> That explains my confusement.
<jtv> And then the tests turned out still to be useful.
<jtv> Damn, I thought I mentioned this in the cover letter.
<wgrant> jtv: Do you know about assertRaisesWithContent
<wgrant> ?
<jtv> Yes
<wgrant> Seems like you could use that in two of those tests.
<jtv> Probably makes more sense to use that.  I'll fix it.
<wgrant> It cuts out about 5 lines from each.
<jtv> Yesâ¦ it doesn't really matter that it's the same exception object, I guess.
<wgrant> No. But if you use getUniqueString you can be pretty sure.
<wgrant> Anyway, apart from that it looks fine.
<wgrant> More publisher tests are always good.
<jtv> Thanks.
<jtv> The change is pushing.
<wgrant> You did sort of mention that it was only test changes in the cover letter, but I sort of assumed that couldn't be the case so ignored it.
<jtv> wgrant: meanwhile, I'm just about through the testing I can do for the new publish-ftpmaster, except I'm not sure it expedites security updates.  Maybe there's more you can think of.
<wgrant> jtv: Not sure that it does, or suspicious that it might not?
<jtv> Suspicious that it might notâexcept we didn't actually get around to publishing a security update, and I don't really know what to look for.
<jtv> Then again, that may not be a blocker.  Can't drag this out forever.
<wgrant> Well, you could just get two fresh uploads, copy one into security and one not, and check the mtime of the security pool and dists files are before the non-security pool and dists files.
<jtv> Thanks..
<wgrant> It's easy enough to check, and I can do it if you want.
<jtv> wgrant: I'd be lying if I'd say I wouldn't want that â if only because you're so much more likely to spot any glaring mistakes than I am.  :)
<jtv> In other words, yes please!
<jtv> (Please imagine better grammar when reading the above)
 * wgrant enters the molasses.
<wgrant> StevenK: :(
<wgrant> StevenK: It wasn't entirely file deletions.
<wgrant> I am disappoint
<StevenK> Haha
<wgrant> jtv: You seem to have a diff in 10-sign-releases.
<wgrant>  #!/bin/sh -e
<wgrant>  
<wgrant> +exit 0 # XXX: DEBUG
<wgrant> +
<jtv> wgrant: that's just because the gpg won't work on dogfood.
<jtv> Without it, test runs will break.
<wgrant> Ah.
<wgrant> Hmm. Why is --post-rsync an option>?
<wgrant> I am loving the sane timestamps in this log.
<wgrant> I might set oneiric to CURRENT for a while to clear out the -updates queue.
<wgrant> NFI how stuff got there :/
<jtv> wgrant: it was an experimental featureâ¦ doesn't look particularly useful on dogfood, though unknown benefits in production.  This way we get to try things.
<wgrant> Ah, k.
<jtv> When you say set oneiric to CURRENT, you mean on dogfood right?  :)
<wgrant> I don't have privileges to do that on production, sadly.
<jtv> If you hadn't added the "sadly," you might still have stood a chance of convincing an admin.  :-)
<wgrant> OK, release, security and partner uploads uploaded.
<wgrant> Now to wait for them to build.
<wgrant> :( partner hates me.
<wgrant> Also, mawson is slow.
<jtv> Have you called the press or should I?
<wgrant> Maybe we can steal gandwana and potassium for staging Soyuz or something.
<jtv> wgrant: meanwhile, do you know the particulars of bug 659769?
<_mup_> Bug #659769: should copy custom uploads to newly-initialised release <derivation> <lp-soyuz> <new-release-cycle-process> <soyuz-publish> <Launchpad itself:Triaged by jtv> < https://launchpad.net/bugs/659769 >
<wgrant> jtv: Unassign.
<jtv> ?
<wgrant> Not really fixable without completely redesigning the custom upload model.
<wgrant> Since they are entirely untracked once the build is processed for the first time.
<wgrant> (they're currently published when a PackageUpload is process by process-accepted, and then nothing is recorded about them)
<jtv> These are PackageUploadCustoms?  Do they get deleted?
<wgrant> They don't get deleted at the moment, no.
<wgrant> But we can't know what is still relevant.
<wgrant> All we can do is republish every PackageUploadCustom ever.
<wgrant> Except for the minor issue that translations tarballs are expired, so that will crash.
<jtv> No other heuristics?  Matching filenames or somesuch?
 * wgrant sobs.
<jtv> Access dates?
<jtv> Status of the attached PackageUploads?
<wgrant> Once they're DONE, they're DONE.
<wgrant> That is the end :(
<jtv> And there's no way to determine that a PackageUpload supersedes an older PackageUpload?
<wgrant> Nope.
<jtv> Access stats on the Librarian file?
<wgrant> There's probably only ever one hit.
<jtv> And they all have different filenames?
<wgrant> Mostly.
<wgrant> wallyworld___: Will you be around to QA your thing in 3 hours?
<wallyworld___> wgrant: sure
<wallyworld___> i've been checking buildbot :-)
<jtv> wgrant: what would that one single hit be?
<wgrant> jtv: process-accepted putting it in dists/
<jtv> And the rest of the use is from there then, I take it.
<wgrant> 2011-08-03 07:33:14 INFO    Processing upload hello_2.6-1df1~querulous1_source.changes
<wgrant> 2011-08-03 07:38:17 INFO    Committing the transaction and any mails associated with this upload.
<jtv> Where do these files go, anyway?
 * wgrant burns Soyuz to the ground.
<wgrant> jtv: http://archive.ubuntu.com/ubuntu/dists/natty/main/
<wgrant> Everything except binary-* and source is custom uploads.
<wgrant> There are also two other types of custom uploads which don't end up there: translations and static-translations.
<wgrant> But those that you see there are basically unpacked tarballs.
<jtv> The translations tarballs are broken up instantly.
<wgrant> (Until a year ago, tarballs unpacked without regard for whether the paths contain ../../ and spray files all over whichever archive the uploader wishes)
<jtv> I remember making one of those code paths check that it's not trying to unpack devicesâ¦
<jtv> But anyway.
<wgrant> Ah yes, translation imports had that issue too.
<jtv> Is there any way to delete custom uploads?
<wgrant> rm -r
<wgrant> The way we currently copy custom uploads is with cp.
<wgrant> Yes, that is mutilating the one-step-from-live archive with cp and rm.
<jtv> Which, admittedly, sounds like fun.
<jtv> Especially with the dot-directory end-of-level baddie.
<wgrant> Hah
<wgrant> lifeless: What are gandwana and potassium up to these days?
<jtv> rm -- rf .<tab><enter><backspace><backspace><ctrl-C><ctrl-C><panic>
<lifeless> wgrant: not sure; possibly going to be the staging scripts machine and achive machine.
<lifeless> [qa]staging that is
<wgrant> Yay
<wgrant> As I was hoping.
<wgrant> Maybe it will happen this decade :)
<wgrant> Because mawson is pissing me off.
<jtv> wgrant: still, do the PUCs really need a redesign or just a bit of extra metadata so we can track and manage them?
<wgrant> jtv: They need a *PublishingHistory table, probably.
<wgrant> As sources and binaries have had since ~the start.
 * jtv discovers it's his turn to sob
<wgrant> I warned you!
<jtv> Still, can't be as bad as BPPH
<wgrant> I wonder if we can do away with scripts/ and cronscripts/
<jtv> wgrant: are the Packages files I see for the installers also custom uploads?
<jtv> They're in binary-* directories, but not at the top level so not clear where they fall.
<adeuring> good morning
<jtv> Hi adeuring
<adeuring> hi jtv!
<wgrant> jtv: Sorry, debian-installer/binary-* are also not custom uploads.
<wgrant> jtv: They're udeb indicies.
<jtv> So it's the translations stuff (which looks like it probably doesn't really need copying), dist-upgrader, and installer-<arch>
<wgrant> Right.
<wgrant> s/translations/ddtp/, to avoid confusion.
<jtv> Where do the versioned directories inside the dist-upgrader and installer-<arch> come from?  Created by hand in the filesystem?  Or do the custom uploads go to a specific version?
<wgrant> The versions are in the custom upload filename, and current is a symlink that we keep up to date, I believe.
<jtv> Looks like it, yes.
<jtv> ISTM we'd only want to copy custom uploads for the current versions.
<jtv> (Which we may not trivially know, unless we have the symlink handy, but it'd make a big dent in the purging problem)
<wgrant> Possibly.
<mrevell> Hello
<jtv> hi mrevell
<jtv> wgrant: looks like it's a bit simpler actually.  I see two kinds of filename: <purpose>_<version>_<arch>.tar.gz and <purpose>_<version>.tar.gz, depending on <purpose>.  I'll bet the last of these for any given (<purpose>, (<arch> or 'all')) in the previous series will take us a long way.
<wgrant> In an extremely distasteful manner, but possibly.
<allenap> wgrant: Thanks for sorting out that rollback.
<jtv> No more distasteful than other places where the debian edifice relies on tarball naming conventions.  Make it (<format>, <purpose>, (<arch> or 'all')) for luck.
<jtv> Check whether the Librarian still has the file, and done.
<wgrant> jtv: We make no assumptions like that outside the custom upload publisher.
<wgrant> Putting that into series initialisation is awful.
<wgrant> Possibly the easiest solution, but still thoroughly awful.
<jtv> Well what produces the tarball names?
<wgrant> Packages.
<jtv> What aspect of packages produces the tarball names?
<wgrant> jtv: It varies.
<jtv> wgrant: taking just the dist-upgrader and the installer, what in the packages determines the tarball filenames?
<jelmer> is canonical.launchpad.utilities.ftests.test_gpghandler.TestImportKeyRing.testHomeDirectoryJob known to be flaky?
<wgrant> jelmer: Yes.
<danilos> stub, about the DB patch, do I assign a patch number myself?
<wgrant> jtv: debian-installer and update-manager
<stub> danilos: Didn't you already have one? Yes, assign it yourself is best.
 * danilos fetches ~launchpad/+junk/dbpatches
<danilos> stub, ack
* StevenK changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: jtv | Critical bugs: 248 - 0:[#######=]:256
<stub> danilos: Just don't use a -0.sql patch number, per newly added comments in the notes of that file.
<danilos> stub, right, reading it now
<jelmer> wgrant: thanks
<jtv> wgrant: those have their own dedicated packageuploadcustomformats?  I think these two are the ones we _really_ care about; the rest at first glance seems more sort of optional.
<wgrant> jtv: They do.
<danilos> stub, do I use the existing "base" number (i.e. 79-0 is assigned, should I use 79-1)?
<lifeless> danilos: https://dev.launchpad.net/PolicyAndProcess/DatabaseSchemaChangesProcess#Patch ids
<stub> danilos: It doesn't really matter. Your patch number needs to be higher than any your patch depends on, but that rarely matters.
<danilos> lifeless, that doesn't help answer my question, but thanks :)
<danilos> stub, ok, thanks, I'll use 79-1 then
<lifeless> danilos: we don't use -0 at all now
<danilos> lifeless, right, stub already explained that (and pointed at comments in the allocated.txt), I was wondering how do I choose the "base" (middle) number for -1, -2 etc.
<lifeless> danilos: could you add something to the file while you are there? I will add a note to the wiki reference I gave.
<danilos> lifeless, sure, I will
<lifeless> thanks!
<jtv> cjwatson: got a moment to discuss possibilities for bug 659769?
<_mup_> Bug #659769: should copy custom uploads to newly-initialised release <derivation> <lp-soyuz> <new-release-cycle-process> <soyuz-publish> <Launchpad itself:Triaged by jtv> < https://launchpad.net/bugs/659769 >
<danilos> stub, can you please run http://paste.ubuntu.com/657714/ (it's the query you optimized yesterday) on qastaging? (if you are busy with the code hosting crisis as well, just ignore me)
<danilos> stub, actually, ignore that, just got a response :)
<lifeless> danilos: hi
<lifeless> danilos: your https://code.launchpad.net/~danilo/launchpad/bug-814580-resubmit/+merge/70262 merge should be against devel - it has no db component
<lifeless> danilos: its only the schema changes that should land on db-devel
<danilos> lifeless, it depends on a branch that has a DB patch
<lifeless> danilos: ah; well you'll need to remember to merge that to devel yourself when the DB patch is deployed.
<danilos> lifeless, how can I know if the DB patch is deployed? devel/stable will contain it or is there something else that I need to watch out for?
<lifeless> danilos: the db patch will be merged to devel by stub / myself after its deployed.
<danilos> (hopefully something more of the push-type instead of a pull type notification)
<danilos> lifeless, ack
<lifeless> I think we will want to mail the list when we do this too, for clarity
<lifeless> eventually there will be a view on the appservers to check, and stub has plans for a report
<danilos> lifeless, it would be even better if original committers are emailed directly as well, perhaps even automatically, but I am sure you guys will figure something smart out :)
<lifeless> first pass is just to get the cycle time benefits
<lifeless> subsequent passes will add polish of various sorts
* danilos changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jtv, danilos | Critical bugs: 248 - 0:[#######=]:256
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #777: FIXED in 6 hr 10 min: https://lpci.wedontsleep.org/job/db-devel/777/
<jtv> Would anyone mind if I removed makeDistroRelease, replacing all calls with makeDistroSeries?
<wgrant> I've been tempted to do that for a Long Time.
<jtv> As a rule, I find that temptation isn't actually much cheaper than indulgence.
<adeuring> lifeless: fancy a 200 diff review? https://code.launchpad.net/~adeuring/launchpad/bug-739052-3/+merge/70272
<adeuring> (yeah, I know it's late for you...)
<lifeless> tis almost early
<lifeless> no diff yet :)
<lifeless> adeuring: you decided not to handle desc, asc, asc, asc, desc as 3 expressions to reduce ?
<adeuring> lifeless: well, I wanted a simple start :)
<lifeless> ok
<adeuring> I know that we could try this too
<adeuring> but I'm wondering if we shouldn't first check how efficient the tuple expressions are
<lifeless> sure
<stub> what is the magic env variable for Storm statement logs again?
<bigjools> LP_DEBUG_SQL
<bigjools> LP_DEBUG_SQL_EXTRA if you want a trace with each one
<adeuring> lifeless: you suggested in an older review to use "resultset.find(resultset.order_by)". Seems that this does not work:
<adeuring> In [11]: rs.find(rs._order_by)
<adeuring> Out[11]: <storm.store.ResultSet object at 0x278a6d0>
<adeuring> In [12]: list(rs.find(rs._order_by))
<adeuring> ProgrammingError: argument of AND must be type boolean, not type integer
<adeuring> LINE 1: ...de_private FROM Bug WHERE Bug.id IN (1, 2, 3) AND Bug.id ORD...
<adeuring>                                                              ^
<lifeless> adeuring: it was pseudo code :P
<adeuring> lifeless: ...but i did not understand it ;)
<lifeless> adeuring: I think I further suggested
<lifeless> find(<initial find clause> + (order by columns tupe) ...
<lifeless> to get both the original data, and the precise data we need
<lifeless> adeuring: however, your exception there is in the where clause not the select clause
<adeuring> lifeless: resultset.find() only allows to extend the WHERE clause, but we can't extend the data returned
<lifeless> adeuring: ah, shame
<lifeless> adeuring: guess we need to fix that too
<adeuring> lifeless: well, I think we can change the original query and used a DecoratedResultSet
<adeuring> that does not require any changes for storm
<lifeless> adeuring: get_select_expr
<lifeless> can  be used too
<adeuring> lifeless: Ah, ok. But is it worth the effort to tweak the result set?
<lifeless> perhaps
<lifeless> I dunno
<adeuring> I mean, DecoratedResultSet are all we need
<lifeless> big picture what I was saying is
<lifeless> - use the existing things to retrieve
<lifeless> - add in the things the order orders on with Asc/Desc removed
<lifeless> - interrogate only those added things
<lifeless> -> we now support queries that return funky stuff, are grouped, etc.
<adeuring> lifeless: ok, I see
<adeuring> lifeless: I suspect that queries with aggregates might fail really bad with StromRangeFactory
<adeuring> unless the aggregates are done in sub-queries
<adeuring> ...sub-selects...
<lifeless> if its in order by
<lifeless> it must be safe to have in the results
<adeuring> well, you could do something like select sum(col) from table order by col; but that's quite pointless...
* jtv changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: danilos | Critical bugs: 248 - 0:[#######=]:256
<lifeless> adeuring: I don't think you can :)
<adeuring> maybe ;)
<lifeless> select sum(col1) from demo order by col1;
<lifeless> ERROR:  column "demo.col1" must appear in the GROUP BY clause or be used in an aggregate function
<lifeless> LINE 1: select sum(col1) from demo order by col1;
<wgrant> wallyworld: Thanks.
<wallyworld> np
<wgrant> allenap: Hi.
<wgrant> rvba: Hi
<allenap> wgrant: Hi :)
<lifeless> adeuring: and if you group by it, you can select it :P
<adeuring> lifeless: yeah
<wgrant> allenap: Bug #809985 needs QA within the next 3 or so hours.
<_mup_> Bug #809985: DistroSeriesDifferenceBaseView modifies shared objects <derivation> <qa-needstesting> <tech-debt> <Launchpad itself:Fix Committed by allenap> < https://launchpad.net/bugs/809985 >
<adeuring> lifeless: back to get_select_expr(): that would work, I think, it it will require two more SQL queries to retrieve the memo values. Do we want this?
<allenap> wgrant: I'm on it already, but thanks for the reminder nonetheless.
<wgrant> allenap: Thanks.
<adeuring> lifeless: argh... i wropte plain nonsense
<lifeless> adeuring: have you had your morning coffee ?
<adeuring> lifeless: not enough... I nearly poured the coffee directly into ash tray this morning...
<lifeless> LOL
<lifeless> well, I have to crash, early start tomorrow.
<lifeless> night all
<adeuring> night lifeless
<wgrant> Night lifeless.
<jtv> hey danilos, I'm going EOD but if you want a bit of fun and you're out of good books: https://code.launchpad.net/~jtv/launchpad/we-prefer-distroseries-thank-you/+merge/70282
<jtv> Something tells me I'll be approving it myself though.  âº
<danilos> jtv, it looks pretty decent considering the diff has not been generated yet
<rvba> wgrant: Hi
<danilos> jtv, and I thought DistroRelease stuff was the *preferred* spelling for DistroSeries
<jtv> danilos: small linguistic note: it looks pretty decent *because* the diff has not been generated yet.  :-)
<jtv> DistroRelease was the old spelling.
<jtv> Mark had that nice 30,000-line branch that did away with it, a few years back.
<wgrant> rvba: You have some pending QA. and we will are hoping to deploy 55 or so revs in a few hours.
<danilos> jtv, heh, ok, if you say so (regarding the linguistic note)
<rvba> wgrant: ok, I'm on it.
<wgrant> Thanks.
<danilos> btw, I wondered about the state of the deployment report, looks pretty weird to me
<jtv> danilos: go on, ask yourself _why_ the diff hasn't generated yet.  :)
<wgrant> danilos: We reverted the last two deployments.
<wgrant> danilos: And found two bad revisions.
<wgrant> Some of which were already deployed.
<wgrant> danilos: We should be good to deploy again in 3.5 hours, and should deploy as soon as it is possible.
<stub> Where is my long poll diff refresher dammit.
<wgrant> danilos: Only one item in the current buildbot run needs QA, and it's quick and documented in the bug, so you should be able to deploy just about as soon as it hits qastaging.
<stub> danilos: Short branch to review - https://code.launchpad.net/~stub/launchpad/trivial/+merge/70284
 * stub wanders off for a bit for kitchen detail
<wgrant> bigjools: If jtv returns, could you inform him that expedited security processing works fine?
<bigjools> wgrant: he's EoD.  But thanks for letting me know.
<danilos> stub, diff is still not showing up :(
<bigjools> I think we're close to releasing this puppy
<wgrant> bigjools: I think so.
<wgrant> bigjools: The core stuff looks good, I believe, but I need to reexamine the hooks at some point.
<bigjools> wgrant: yeah
<LPCIBot> Project devel build #942: STILL FAILING in 5 hr 49 min: https://lpci.wedontsleep.org/job/devel/942/
<danilos> diff generation seems dead
<wgrant> danilos: See #launchpad-ops
<wgrant> And see the spam that should be in your inbox :)
<danilos> wgrant: thanks (and I am only trying to review stuff, thus not getting the spam)
<wgrant> Heh.
<wgrant> We'll hopefully have nagios checks on scriptactivity soon, but until then someone should watch for the emails.
<danilos> wgrant: right, it's one thing we are particularly bad at (watching all the -errors emails)
<danilos> stub, r=me, with a minor proposal which is entirely up to you (if you perhaps want to still update all of bug heat values with a single query, but you know better if that's worth it :))
<StevenK> wgrant: Er, I think you broke devel
<StevenK> wgrant: https://lpbuildbot.canonical.com/builders/lucid_lp/builds/1212/steps/shell_6/logs/summary
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/i-suck/+merge/70298
<wgrant> StevenK: No diff, due to the ongoing incident, but you can get it from loggerhead.
<stub> danilos: ta
<StevenK> wgrant: Right. Approved, please land
<wgrant> I forgot that |nothing swallows exception.
<StevenK> danilos: You are the last person who needs to do QA, r13586
<danilos> StevenK, hasn't that been QAd already?
<danilos> StevenK, oh, separate db-devel landing has made qa-tagger mess up my tags, we are all good
<StevenK> danilos: Until a bunch of new revs hit stable. Thanks!
<jml> the diff generation issue wasn't announced on identi.ca :(
<danilos> StevenK, thankfully, forthcoming testfix will put a stop to that! ;)
<StevenK> danilos: Which wgrant has already fixed.
<danilos> StevenK, he should be sleeping, shouldn't he? :)
<StevenK> danilos: He fixed it before his bed time :-P
<henninge> abentley: https://wiki.canonical.com/IncidentReports/2011-08-03-LP-failing-jobs
* danilos changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 248 - 0:[#######=]:256
<mtaylor> hey guys! did the bzr haproxy timeout get changed back down again?
<jelmer> hi mtaylor
<mtaylor> hi jelmer !
<jelmer> (checking the haproxy status)
<mtaylor> jelmer: I'm getting timeouts
<jelmer> mtaylor: this is when a connection is idle for more than X seconds?
<mtaylor> jelmer: not full sure - still tracking down on my side
<jelmer> mtaylor: I think I saw your bug report yesterday about bzr handling that more gracefully
<mtaylor> jelmer: https://jenkins.openstack.org/job/nova-tarmac/109707/console
<mtaylor> jelmer: if you look at the bottom there, you'll see the "Connection to bazaar.launchpad.net closed by remote host."
<LPCIBot> Project devel build #943: STILL FAILING in 5 hr 52 min: https://lpci.wedontsleep.org/job/devel/943/
<sinzui> jcsackett, do you have time to mumble?
<jcsackett> sinzui: i will in five minutes.
<lifeless> morning
<sinzui> jcsackett, https://code.launchpad.net/~sinzui/+recipe/mmm-archive-manager-daily
<lifeless> bac: hi
<bac> hi lifeless
<lifeless> bac: just a very small note - it was great to see you identifying some dupes; could you please bias towards duping on the oldest one ?
<bac> lifeless: i would've but i was already working the other so i biased toward it
<lifeless> bac: ah... so I've just messed things up for you by switching them around
<lifeless> bac: I'll put them back.
<bac> perhaps
 * bac will live
<lifeless> bac: this isn't really documented anywhere, but I've a convention with timeouts of puting the page id in the title, it helps folk searching for the bug given a OOPS summary report.
<lifeless> in this case I failed to update the subject on the newer one, and in not doing that I failed to notice it was a dupe :) - mea culpa
<lifeless> there, all back, and I've copied the subject over
<lifeless> sorry for the interrupt!
<bac> lifeless: np!
<sinzui> jcsackett, I am jinxed. The battery indicator just went belly up
<sixstring> Howdy, #launchpad-dev. I'm happy to join the crew working on launchpad.
<SpamapS> lifeless: here's a bug that consistently times out .. I suspect its a known issue: https://bugs.launchpad.net/ubuntu/+source/gnome-settings-daemon/+bug/804896
<_mup_> Bug #804896: gnome-settings-daemon assert failure: gnome-settings-daemon: ../../src/xcb_io.c:140: dequeue_pending_request: Assertion `req == dpy->xcb->pending_requests' failed. <apport-crash> <i386> <oneiric> <unity-2d> <gnome-settings-daemon (Ubuntu):Triaged by rodrigo-moya> <gnome-settings-daemon (Ubuntu Oneiric):Triaged by rodrigo-moya> < https://launchpad.net/bugs/804896 >
<lifeless> sixstring: welcome
<lifeless> whats the OOPS id
<jml> sixstring: \o/
<jml> sixstring: welcome :)
 * sixstring is running rocketfuel-setup.
<sixstring> Yep. It's a long process. :)
<sixstring> Esp. at this coffeeshop.
<jml> I'll bet!
<sixstring> ~300KB/s. I'd better go get another cup o' joy.
<sixstring> Incidentally, I don't suppose rocketfuel-setup would recover if I halted the bzr checkout step, would it?
<lifeless> sixstring: not well :)
<sixstring> 183MB so far, so good. Guess I'll just wait it out. :)
<Ursinha> hello launchpadders :)
<Ursinha> is there a way to bring someone's attention to a bug other than subscribing them to that?
<lifeless> email them a url ?
<Ursinha> lifeless: I'll pretend you didn't suggest me that :P
<Ursinha> lifeless: what I see people doing is assigning to the bugtask the one they want input from
<Ursinha> that's misleading
<Ursinha> (from a defect analyst point of view)
<lifeless> Ursinha: if the bug is blocked on that persons action, it seems accurate to me.
<Ursinha> lifeless: right
<lifeless> but there is no 'ask for input from <person>' facility in LP itself.
<Ursinha> so assignee not just the one who's going to actually fix, but the one the bug is depending on... ok, makes sense
<Ursinha> I'm working on the ubuntu server bug triage process, so I saw that a couple of times
<Ursinha> people don't actually follow bug reports when there are lots and lots of them
<Ursinha> well, you do, but that's you :P
<lifeless> :)
<sinzui> stevek, mumble
<LPCIBot> Project devel build #944: STILL FAILING in 5 hr 52 min: https://lpci.wedontsleep.org/job/devel/944/
<sinzui> wallyworld, ,maybe the save event needs to explicitly set the NUM_SEARCHES to 0
<wallyworld> sinzui: perhaps, i'll look at that. i also want to see when it went bad
#launchpad-dev 2011-08-04
<wallyworld> sinzui: yes, looks like it was r13573, bug 809508
<_mup_> Bug #809508: Suggest branches with current bug number when associating a bug with a branch. <qa-ok> <Launchpad itself:Fix Released by benji> < https://launchpad.net/bugs/809508 >
<wallyworld> commit msg: Make the search box stay disabled and the spinner visible until all outstanding searches are finished
<wallyworld> mp was self reviewed, so something slipped through there
<wgrant> wallyworld: Should we abort the rollout?
<StevenK> wallyworld: If you're unclear, just revert it
<mhall119> does anybody else get an error on this page: https://api.launchpad.net/1.0/~ubuntu-br/homepage_content ?
<mhall119> wgrant: ^^
<mhall119> or lifeless
<mhall119> whoever is awake
<mhall119> cjohnston: this is the cause of our lpupdate failure I think
<StevenK> That doesn't look like JSON to me
<cjohnston> hmm.. where is that coming from
<mhall119> it's part of the response from https://api.launchpad.net/1.0/~locoteams/members
<mhall119> that part of the json, brazil's homepage_content
<mhall119> is where our script if throwing exceptions
<cjohnston> so something is wrong it looks like on LPs end in the Homepage Content area
<mhall119> possibly
<mhall119> I'm still not quite sure what though
<cjohnston> I wander if its something with the localization of it
<mhall119> I think it's more likely that it's not specifying that it's unicode
<wgrant> StevenK: Your browser Accepts HTML, so it will give it HTML.
<StevenK> wgrant: Even wget doesn't show JSON
<wgrant> It does.
<wgrant> Well, curl does.
<wgrant> "Usu\u00e1rios do Ubuntu no Brasil.\n\nTime para usu\u00e1rios do Ubuntu no Brasil.\n\nComece Aqui:  -  http://www.ubuntu-br.org/comece\n\nP\u00e1gina Principal:  -  http://www.ubuntu-br.org\nWiki:  -  http://wiki.ubuntu-br.org\nPlaneta:  -  http://planeta.ubuntu-br.org\nF\u00f3rum  -  http://ubuntuforum-br.org/\nDocumenta\u00e7\u00e3o:  -  http://wiki.ubuntu-br.org/Documentacao\n\nAntes de entrar neste time, voc\u00ea deve ler, concordar ...
<wgrant> ... e assinar o C\u00f3digo de Conduta do Ubuntu (http://www.ubuntu-br.org/codigodeconduta)."
<wgrant> wget does too.
<wgrant> Content-Type is correctly "application/json"
<wgrant> No encoding is required, as it's ASCII.
<wgrant> mhall119: What's the error your script is throwing?
<mhall119> yeah, I'm getting json
<mhall119> wgrant: http://paste.ubuntu.com/658298/
<wgrant> I can access the members collection without trouble.
<mhall119> I've tracked it down so simplejson trying to decode a string
<wgrant> Is it possible you've got a corrupt cache?
<wgrant> Try removing your launchpadlib cache.
<mhall119> I don't have access to do that on cranberry
<mhall119> and my local dev isn't even getting me past lp_login, not sure why
<mhall119> ok, I just need to get my local dev working to the point where I can watch this with a debugger
<wgrant> I doubt it will happen locally.
<cjohnston> mhall119: there is a vangard on
<mhall119> cjohnston: I've already got them doing it
<cjohnston> cool
<mhall119> now we wait until 10pm
<mhall119> the cron should have run again by then
<LPCIBot> Project devel build #945: STILL FAILING in 5 hr 48 min: https://lpci.wedontsleep.org/job/devel/945/
<jtv> wgrant: did you have any luck trying the new publish-ftpmaster yesterday?
<wgrant> jtv: It finished not long after you EODed.
<wgrant> jtv: Expedited security processing works fine.
<jtv> You have made me so happy.
<wgrant> I'm pretty happy with the whole thing now, but I need to check over the hooks a bit.
<jtv> I'd appreciate that â as soon as you're satisfied with them, we can make the switch.
<wgrant> jtv: They all look pretty sensible.
<wgrant> What's the worst that it could do...
<jtv> *cough*
<jtv> wgrant: then I shall proceed to move this towards productionâ¦
<jtv> â¦albeit with some trepidation.
<wgrant> *immense
<wgrant> But I think we can do little else now.
<adeuring> good morning
<mrevell> Hi
<jtv> ah yes, bigjools: when copying custom uploads from a previous Ubuntu series, I don't suppose the pocket matters much?
<bigjools> jtv: nup
<jtv> Argh.  But the changes file does.  It'd be nice if I could just copy the LFA id, but that's not currently supported.
<jtv> DistroSeries.createQueueEntry expects to have to store the changes file into the Librarian.
<wgrant> You are creating new PUs and PUCs? :/
<jtv> wgrant: Have to.  What's the problem with that?
<wgrant> you don't have to; it's just easier than the rather messy and time consuming proper fix.
<wgrant> Perhaps understandable.
<wgrant> But definitely another pile in the hack pile.
<wgrant> another hack in the hack pile
<wgrant> That's the one.
<wgrant> It's like delayed copies, except even worse defined :(
<jtv> Internet connections: you just can't trust 'em.
<jelmer> Is staging used to test anything in particular ? I wonder if now would be a good moment to get the  branch with the newer bzr deployed there for the extended QA discussed on the list.
<wgrant> jelmer: DB patches (at least for the next week) and mailing lists and some build farm stuff.
<wgrant> Pretty much everything else is on qastaging these days.
<jelmer> wgrant: Ah, cool - thanks.
<jelmer> Time to land my branch and QA it then.
<wgrant> jelmer: Land it? Do you mean get it cowboyed onto staging?
<wgrant> Also, staging is having a bit of evil performed on it a bit the last couple of weeks, for testing fastdowntime.
<wgrant> Not sure if stub has much more of that planned.
<jelmer> ah, hmm
<lifeless> wgrant: waiting on pgbouncer just now, for prod.
<wgrant> lifeless: We're going with the 3m downtime?
<lifeless> wgrant: stub has it down to ~2m
<lifeless> wgrant: slony deletes and creates ~ 500 triggers on all nodes.
<wgrant> I had a 2-slave no-op update down to 45s here by skipping trusted.sql, IIRC.
<wgrant> The master takes around 12 seconds.
<wgrant> So the triggers don't take tooooo long.
<lifeless> we may be seeing cold cache table metadata on staging timings
<stub> I'm still looking at shaving time off. Skipping trusted.sql if unmodified is doable - just need to store a hash of the file in the db somewhere.
<wgrant> stub: Can't we treat trusted.sql like the rest of the schema?
<wgrant> Have a base + patches?
<stub> And we can deprecate updating it
<wgrant> lifeless: Surely the metadata should be pretty tiny.
<stub> wgrant: We already have had to with the bugsummarywork
<jelmer> wgrant: I'm wondering with whom to coordinate this exactly, is there a point of contact/process?
<jelmer> stub: Are you planning to do more testing on staging for fastdowntime?
<stub> jelmer: fastdowntime is live on staging, any more tests are just improving the process.
<wgrant> stub: Right, so let's do that everywhere and stop recreating all those functions each time :)
<stub> wgrant: The nice thing about trusted.sql is you can edit your code in place, get diffs etc. Current approach by punting that to the dbpatches means cut & paste to tweak a stored procedure, and locating the current implementation involves grep, and source will eventually disappear when we clean out the old db patches. Not sure if we can easily solve any of the downsides, but thoughts welcome.
<wgrant> stub: Agreed, and that really hurt during the bugsummary debacle.
<wgrant> But reapplying trusted.sql is really slow.
<stub> (cycling the baseline will need some more thought too, as it is tied up with trusted.sql, but I need to fix that already)
<stub> wgrant: Yup. It seems to be a per-statement overhead in there somewhere, which is why I already dropped comments.sql out of patch application.
<stub> I could be evil and package trusted.sql up in a stored procedure that applies the contents :-)
<stub> your timing on trusted.sql is good though - I hadn't gone to that granularity yet
<wgrant> I forget exactly, but I think removing trusted.sql saved around 40s. But it only started at around 100s, so I'm not sure what's up with staging.
<wgrant> make -C database/replication devsetup is awesome, btw.
<LPCIBot> Project devel build #946: STILL FAILING in 5 hr 56 min: https://lpci.wedontsleep.org/job/devel/946/
<wgrant> stub: What intervals are staging's slons running at?
<wgrant> The variance gets a lot lower if I reduce the no-SYNC interval to 2s from 10s.
<wgrant> 39-41s without trusted.sql, rather than 45-55s.
<wgrant> Which I guess makes sense.
<wgrant> Since it's waiting for synchronization, with no replicatable write activity.
<wgrant> And if it's waiting for an extra sync on both sides of trusted.sql, that could explain much of the difference.
<wgrant> Nope, adding trusted.sql adds 45s even with 2s forced syncs.
<stub> wgrant: I landed a branch last night that drops staging down to 1 second syncs and 5 second forced syncs to see if it changes things significantly.
<stub> 1 second sync check, 5 second forced sync
<wgrant> stub: What was it before?
<wgrant> We may want to push through some update right before each sync.
<stub> 2 second sync check, 10 second forced sync (defaults)
<wgrant> Ah.
<stub> I don't think it will be significant - maybe shave 10 seconds off, or 15 if we drop the sync poll time to 0.5.
<stub> But worth testing in case.
<wgrant> Adding a third slave has added 15s to the trusted.sql time...
<wgrant> Which is about right.
<wgrant> 12-14s to apply to the master, then apparently a similar time to each slave, serially?
<wgrant> I'm still not entirely clear on how the DDL gets propogated.
<wgrant> I guess I should read more on that.
<wgrant> Blah, but it slows down the no-trusted.sql upgrade by nearly 4s too.
<wgrant> I guess that matches up fairly well, but is still disappointing.
<wgrant> Particularly if prod has ... 8 replicas?
<wgrant> Maybe 6.
<wgrant> I can't count.
<jtv> wgrant: I'm looking into where to copy the latest custom uploads to a new distroseries.  Julian suggests I might use the hook we already have for the case where a distroseries is being published for the first time.  What do you think?
<wgrant> jtv: I suggest you do whatever is quickest and easiest to remove when we fix custom uploads properly. That may be that.
<wgrant> But it's possibly appropriate to put it right in IDS, if you're creating new PUCs.
<wgrant> Since it doesn't need to manipulate the disk directly.
<jtv> But it also doesn't currently use DSP.
<wgrant> DSP?
<wgrant> DistroSeriesPArent?
<jtv> Yes.  Well, the code I have now doesn't actually care whether you use it or not; but the initial plan is to do this based on previous_series.
<jtv> Or whatever the old parent_series got renamed to.
<wgrant> Right, that sounds somewhat sensible.
<wgrant> Still more appropriate for IDS IMO, if you're creating PUCs.
<jtv> Does IDS get run just once per derived series?
<wgrant> Yes.
<bigjools> yes, but even creating PUCs is not enough, you need to force them through process-accepted
<wgrant> You'll need to create PUs and PUCs, and process-accepted will handle them in its own special way.
<bigjools> it makes me shudder
<wgrant> I believe I've expressed my opinions on process-accepted and this new strategy.
<bigjools> least-worst option for now.  We don't have time to redesign this
<wgrant> Yes.
<wgrant> It is terrible, but probably least-worst.
 * bigjools observes the ivory towers in the distance
<stub> wgrant: An interesting thing about trusted.sql is that it doesn't need to be applied via slonik at all
<wgrant> stub: It shouldn't have to be, no, so we could possibly work around it.
<wgrant> But it may still be slow without it.
<stub> yes, but I'm looking at 2 minutes. If I can shave 40 seconds off that, it is a big win percentage wise.
<stub> It does mean it is being applied outside the db patch transaction
<wgrant> The no-op upgrade SQL (just the two UPDATE LDR statements) takes 15s here when executed directly through slonik with three slaves.
<wgrant> So we have 30s of overhead in the script.
<wgrant> Probably the sync on either side.
<jtv> bigjools: I need to force my copied PUCs through process-accepted?  Won't that run anyway?
<wgrant> jtv: Yes.
<wgrant> Just create them on new ACCEPTED PUs, and all will be good.
<bigjools> jtv: it's ok as long as there's a PU in ACCEPTED state
<wgrant> Well, bad, but it will work.
<bigjools> it is a horrible hack but then custom uploads are a horrible hack :(
<jtv> So I have to create my PUs as ACCEPTED?
<bigjools> yes
<stub> Given we have already applied stuff from trusted.sql live, I guess we can live without it being applied in-transaction with the db schema updates
<wgrant> jtv: Maybe just create them and then pu.setAccepted.
<bigjools> that will also work
<wgrant> Er.
<wgrant> No, there's setDone... don't think there's setAccepted.
<wgrant> Just acceptFrom*
<wgrant> But one of those might work.
<wgrant> stub: Hm.
<wgrant> stub: trusted.sql applies in 100ms directly.
<wgrant> Let me force it through slonik and see what happens...
<wgrant> 13s.
<wgrant> WTF
<wgrant> So something is causing it to execute all the syncs serially, or something.
<wgrant> Where that doesn't happen with the no-op patch SQL.
 * wgrant tries merging them into one file.
<wgrant> With the minor issue that they already end up as one file.
<wgrant> So where is the extra 45s coming from.
<wgrant> Blaaah.
<wgrant> The 13s overhead is on each execute_script.
<stub> wgrant: When we check for sync, we wait for the sync to be confirmed by ALL nodes. So that blocks until a node has finished doing something like apply a patch and it gets around to acking.
<wgrant> stub: Yeah, but the patches take <100ms each.
<wgrant> So either the slon goes to sleep for 15s after applying the patch, or something else is happening.
<wgrant> stub: Overhead vanishes if I change upgrade.py's run_sql to merge it all into one big file, with a single execute_script.
<stub> wgrant: I'm thinking on why patches sometimes get serialized, rather than applied on slaves simultaneously. I think slave2 might block on slave1 saying 'sure - go ahead' because slave1 has already started processing a dbpatch.
<wgrant> Well, it's down to below 2s, at least.
<stub> cool
<stub> we don't do that already?
<wgrant> No.
<stub> oh... one execute script
<wgrant> We create one slonik script.
<wgrant> With multiple SQL files, and multiple execute_script stanzas.
<wgrant> It looks like the syncs around the second one end up serialised, somehow. Might watch slon logs tomorrow.
<wgrant> Which is really going to hurt production, if it's 16s per slave rather than 3s.
<stub> so we want an assert in there that len(open(script).readlines()) < 1000, or we overflow a slony constant I think set at compilation time
<wgrant> dafuq?
<wgrant> Seriously?
<stub> yup
<stub> we are weird. agile is foreign to most db environments and dbas
<stub> Our sort of automation tends to scare the bejesus out of people.
<wgrant> stub: Are the syncs in execute_script insufficient?
<wgrant> We currently seem to sync on either side.
<wgrant> Which is an extra $while.
<wgrant> Overhead in upgrade.py is still 30s.
<wgrant> Will be more on prod.
<wgrant> But I wonder if you can try the upgrade.py SQL merge thing on staging?
<wgrant> See how far down it takes it.
<wgrant> Even better if you can reduce the forced sync interval.
<stub> wgrant: Sure. Stick in a MP for db-devel (we want the fastdowntime stuff on that branch still)
<stub> I have no problem lowering the sync check and forced sync times if it helps. It won't affect our load.
<wgrant> As you say, it should only save a few times the reduction. Might get better numbers tomorrow when I am awake.
<wgrant> ./src/parsestatements/scanner.h:#define MAXSTATEMENTS 1000
<wgrant> Kill me now.
<wgrant> aaaaaaa
<wgrant> This is not sensible!
<wgrant> ./src/parsestatements/scanner.c:int STMTS[MAXSTATEMENTS];
<wgrant> D:
<wgrant> malloc is hard, I guess.
<stub> wgrant: Shouldn't be a genuine problem given we won't want a backlog of patches to apply in one hit.
<stub> wgrant: esp if we decide to pull trusted.sql out
<wgrant> $ wc -l /tmp/trusted.sql
<wgrant> 2111 /tmp/trusted.sql
<wgrant> I guess there are some nice long statements in that.
<wgrant> Will be fun counting them.
<stub> yes - a whole stored procedure counts as '1'
<stub> oic
<wgrant> Yeah.
<stub> You are counting semi colons out side of strings, which you can assume are 'string' or $$string$$. Or just ignore the problem for now.
<wgrant> I think it will be ignored for now.
<wgrant> Will see how upgrade.py copes if it's violated.
<wgrant> I mean, it will probably cause slon to segfault or something...
<wgrant> No biggie.
<stub> it will barf in the transaction and roll back.
<stub> meybe picked up at slonik compilation time
<wgrant> Hah, all my logging changes conflict with yours.
<stub> comments.sql will make it explode :)
 * stub disappears for an hour
<wgrant> stub: https://code.launchpad.net/~wgrant/launchpad/single-execute_script/+merge/70432
 * wgrant sleeps.
<jelmer> g'night wgrant
<gary_poster> abentley, when you are around, if you could let me know which of bugs 820510, 820511 and 820516 are worthy of critical, or help me decide at least, maybe yellow squad could take one or more of those.
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 247 - 0:[#######=]:256
* allenap changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: allenap | Critical bugs: 247 - 0:[#######=]:256
<abentley> gary_poster:  820511 and 820510 give the best bang for buck, but I don't think either one is truly critical, and marking them critical makes it hard for us to see what's truly critical.
<henninge> abentley, adeuring: I have to switch locations now. Should be back in time for the stand-up.
<abentley> henninge: cool.
<gary_poster> abentley, ok thanks.  "hard to see what's truly critical"...I don't think it's any different than the existing situation with the critical bug tag, but let's not go there.  Practically, do you agree with wgrant that this is something that should be tackled now, even when our focus is supposed to be on "critical" bugs?
<abentley> gary_poster: I think it makes sense to tackle soon.  I don't think the focus on "critical" bugs is meant to exclude us working on anything else.
<gary_poster> abentley, ok.  I'll encourage yellow to grab at least one of those, asking you for guidance when we get there.
<gary_poster> thanks
<abentley> gary_poster: np.
<StevenK> allenap: O HAI!
<allenap> StevenK: HAI!
<allenap> Whassup?
<StevenK> allenap: You marked https://code.launchpad.net/~stevenk/launchpad/populate-bprc/+merge/69412 as Needs Fixing a week ago, and you have since been smacked down by wgrant and lifeless. Can you reconsider? :-)
<allenap> StevenK: There were other points in the review :)
<StevenK> allenap: Right, so I'll look at getCandidateBPRs(). I disagree about the loop size -- if it gets killed half-way through a loop, for example?
<StevenK> allenap: And I don't agree the test is dependant on test data. How?
* jcsackett changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: allenap, jcsackett | Critical bugs: 247 - 0:[#######=]:256
<allenap> StevenK: Why do you only want to do one thing round the loop? I suppose it keeps transactions short, but it's meant to tune for than anyway isn't it?
<StevenK> allenap: So, each iteration reads a .deb from the librarian, parses the contents and starts tossing rows into BPRC and BPP. If that loop gets interruptted, do I end up with half of the data in the tables and half not?
<allenap> StevenK: No? Why is that a concern? I don't think it interrupts in the middle of a run. And transactions.
<StevenK> allenap: My basis for this was the PSC work -- and that was actually unpacking source packages and dealing with a bunch of temporary files, so it was more critical it wasn't killed in the middle of a run
<StevenK> allenap: Conceeding the loop tuning question -- I'll bump it 20 or so
<allenap> StevenK: Okay :)
<StevenK> allenap: So, the test is dependant on sample data?
<allenap> StevenK: It looked that way, but I didn't dig much. If you say it's not then that's all I want to know.
<allenap> StevenK: What about 3? That doesn't actually look like it's going to work how it is.
<StevenK> allenap: I didn't want to include another .deb in the tree, so I make use of one that's already there due to archiveuploader tests
<allenap> StevenK: Oh, ignore me. EPARSE.
<allenap> StevenK: So, r=me now, but consider point 4.
<StevenK> You'd prefer self.layer... ?
<StevenK> Wait, ignore me, I can't read.
<StevenK> allenap: -1 for including the IRC log in the MP :-P
<allenap> StevenK: I just added a tl;dr. What's wrong with an IRC log?
<StevenK> allenap: Just teasing :_P
<allenap> :)
<rvba> allenap: I've fixed my MP, could you add it to your queue? https://code.launchpad.net/~rvb/launchpad/bug-820900/+merge/70434
<allenap> rvba: Sure.
<rvba> allenap: Thanks.
<allenap> rvba: r=me
<rvba> allenap: \o/
<cjohnston> thanks mrevell
<mrevell> Hi cjohnston
<mrevell> no problem at all cjohnston :) thanks for your help
<sinzui> jcsackett, can you review https://code.launchpad.net/~sinzui/launchpad/person-picker-expand-1/+merge/70459
<sinzui> The MP is lying about the changes from the prerequisite branch. I attached the actual changes
<jcsackett> sinzui: thanks, i was just looking at the MP diff and getting confused about the scope of the changes. :-)
<jcsackett> sinzui: r=me.
<sinzui> jcsackett, thank you.
<jcsackett> sinzui: you're welcome. :-)
<nigelb> zomg.
* allenap changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: jcsackett | Critical bugs: 247 - 0:[#######=]:256
<nigelb> err, wrong channel :P
<LPCIBot> Project devel build #947: STILL FAILING in 5 hr 53 min: https://lpci.wedontsleep.org/job/devel/947/
<abentley> jcsackett: Could you please review https://code.launchpad.net/~abentley/launchpad/induce-latency/+merge/70471
<jcsackett> abentley: sorry, i missed your msg; i'm looking at the MP now.
<abentley> jcsackett: cool.
<jcsackett> abentley: looks good to me, r=me.
<lifeless> bug 820510
<_mup_> Bug #820510: hard to turn on extra logging for twisted job runner <jobs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/820510 >
<sixstring> So, I killed rocketfuel-setup yesterday. I think it was right in the middle of the "bzr pull" step. Now, when I run "rocketfuel-get", I get "ERROR: No WorkingTree exists".
<sixstring> BTW, which pastebin do we use here?
<lifeless> any that ou want
<sixstring> How do I fix my bzr repo? (I'm familiar with hg and svn, but I'm a bzr n00b.)
 * sixstring searches for paste.lifeless.org. ;)
<sixstring> http://www.pastie.org/2321492
<lifeless> put set -x at the top of rocketfuel get so you can see the failing bzr command
<sixstring> It's failing at "bzr up ~/launchpad/lp-sourcedeps/download-cache".
<sixstring> In hg world, I'd just blow away that directory and re-clone it. Any bzr advice?
<sixstring> BTW, the bash mojo in rocketfuel is impressive.
<sixstring> OK, I'll try this: (1) mv ~/launchpad ~/launchpad-busted (to get it out of the way) then (2) re-run rocketfuel-setup.
<sixstring> That seemed to work.
<sixstring> I don't suppose there's a log-bot logging this stuff for posterity?
* jcsackett changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 247 - 0:[#######=]:256
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #948: FIXED in 5 hr 50 min: https://lpci.wedontsleep.org/job/devel/948/
<wgrant> wallyworld: You said you had a better bzr plugin?
<wallyworld> wgrant: yes, i'll email it to you
<wgrant> Thanks. Going to submit it upstream?
<poolie> ooh, for what?
<wgrant> poolie: PyCharm.
<poolie> that'd be good to send it up to them
<poolie> if it has significant new features maybe we can put it on the blog?
<wgrant> wallyworld: Also, do you have any hints on setting up the project? You said to create it outside the branch, but I'm not sure how to get everything into it.
<wgrant> The original is already on LP (https://launchpad.net/bzr4j)
<lifeless> wgrant: do you know why sinzui marked bug 345349 invalid ?
<_mup_> Bug #345349: Notifications about private branch linkages sent to unprivileged subscribers <disclosure> <lp-bugs> <Launchpad itself:Invalid> < https://launchpad.net/bugs/345349 >
<wallyworld> wgrant: the email may help. my copy of the bzr plugin has changes which i've submitted to the lp project but which have not yet been merged in
<wgrant> lifeless: No, and the call finished like 2 minutes ago :(
<wallyworld> wgrant: read the email first and then ping me with any questions
<wgrant> wallyworld: Indeed, that helps a lot, thanks!
<wallyworld> wgrant: np. it's a very terse summry. there's potentially a lot more you may need help with. but see how you go....
<wallyworld> wgrant: also, i have debuging working when running lp inside an lxc container
<lifeless> wallyworld: oh nice
<lifeless> wallyworld: what did you do to make that work ?
<wallyworld> you have to add a couple of lines to the top of bin/run
<wallyworld> and tweak a setting inside the debug config of pycharm to tell it to connect to a "remote" instance
<wallyworld> lifeless: you did realise we were talking about pycharm? i'm not sure about debugging using pdb etc
<lifeless> wallyworld: yes, I did
<wallyworld> cool, just checking :-)
* wallyworld changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: wallyworld* (jtv) | Critical bugs: 247 - 0:[#######=]:256
<wgrant> wallyworld: Should I create a separate empty directory for the project?
<wgrant> I can't find a way to add external stuff.
<wallyworld> wgrant: yes, in my case, i created a ~/PyCharmProjects directory and saved the lp project there
<wallyworld> then you can got o settings->project structure and add a content root
<wallyworld> the content root is your working tree
<wgrant> Ahh, thanks.
<wallyworld> and from there you mark stuff as sources and exclude things as per my screenshots
<wgrant> Yup.
#launchpad-dev 2011-08-05
<wgrant> wallyworld: It looks like you can use PyCharm's buildout support now, instead of specifying the eggs manually.
<wallyworld> wgrant: yes, i hadn't gotten around to looking at that yet :-(
<wallyworld> maybe i should make the time
<poolie> i'm getting a timeout setting bug status, https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-2043DR6
<wgrant> wallyworld: I had to use bin/i18nextract. Only bin/i18n* seem to have the necessary paths.
<wgrant> wallyworld: But I added that and removed all the eggs from the sources config, and it seems to work.
<wallyworld> excellent!
<wgrant> And, indeed, under "External Libraries" there is now "Buildout Eggs"
<poolie> wallyworld: which jvm do you use for pycharm?
<wallyworld> wgrant: one extra thing - if you want pycharm to know about what's inside bin/run, bin/test etc, open those once initally and tel pycharm they are python files. pycharm relie son file extensions and thouse don't have any :-(
<wallyworld> poolie: i use sun's jdk
<poolie> is that packaged in ubuntu?
<wallyworld> yes, can't recall whether main or ....
<wgrant> It's in partner.
<wallyworld> makes sense
<poolie> ok, got it
<StevenK> wgrant: Do you think it's safe to remove bazaar-experts now?
<poolie> urh jb seems to have a lot of trouble with unity on my machine
<lifeless> jim bean ?
<lifeless> wgrant: -haaaaaaaaaaaaa- pids within lxcs are not unique across lxcs.
<wgrant> lifeless: Nope.
<wgrant> StevenK: Probably, yes.
<StevenK> lifeless: Can haz bazaar-experts killed?
<lifeless> wgrant: how does our test config isolation work .. :P
<wgrant> I wasn't sharing a working directory.
<wgrant> But I guess if you do then that's a problem :/
<lifeless> wgrant: you made one per parallel thread ?
<wgrant> lifeless: The LP tree lived in the base container.
<wgrant> It was not bind-mounted it.
<lifeless> ah
<wgrant> s/it/in/
<lifeless> anyhow, memcached sorted.
<lifeless> the rabbit layer took a -long- time to dies
<lifeless> but no longer hanging
<lifeless> still seeing the odd
<lifeless> could not get IP address - aborting.
<lifeless> with containers taking too long to startup :(
<lifeless> and
<lifeless> No handlers could be found for logger "lazr.smtptest"
<StevenK> That one you see on buildbot and ed2 too
<StevenK> *ec2
<lifeless> at least now the layers are all starting, seem to be getting clean subunit
<lifeless> FAIL: canonical.testing.ftests.test_layers.ZopelessTestCase.testMemcachedWorking
<lifeless> Difference: True != False: memcached is live but should not be.
<lifeless> TransportError: Transport error: [Errno 1] Operation not permitted: '/var/tmp/bazaar.launchpad.dev/mirrors' [Errno 1] Operation not permitted: '/var/tmp/bazaar.launchpad.dev/mirrors'
<sixstring> Anyone know where the source for login.launchpad.net is located? I'm looking for the Django project. (And that's not what launchpad itself is...Zope!)
<wgrant> sixstring: lp:canonical-identity-provider
<lifeless> lp:canonical-identity-provider
<sixstring> Thanks!
 * sixstring tosses some donuts toward wgrant and lifeless.
<lifeless> ok, that seems good now
<lifeless> id=3, tests=2089, failures=5, skips=80
<wgrant> lifeless: Nothing hung?
<lifeless> wgrant: indeed
<wgrant> Full test suite run now.
<wgrant> Now!
<lifeless> wgrant: patience kemosabe
<StevenK> lifeless: Do you need changes to hit devel before we turn parallel-test back on?
<lifeless> I'm comparing single threaded on laptop with 8 threads on desktop
<lifeless> StevenK: no, but I need you to upgrade jenkins to oneiric and setup lxc.
<StevenK> ... I can see that happening with buildbot :-P
<lifeless> hmmm, I wonder if I have the timing fixed branch of testtools here, I seem to be load balancing the tests poorly.
<lifeless> or maybe there is some gargantuan test matching the regex 'canonical'
<lifeless> 7/8 workers completed a while back
<lifeless> or maybe they are running too many tests, but the load-list stuff was fairly robust I thought
<wgrant> There is a test that hangs on 2.7 matching 'canonical', but I guess your containers are lucid and 2.6?
<lifeless> right
<lifeless> its busy in CPU
<LPCIBot> Project db-devel build #784: FAILURE in 5 hr 45 min: https://lpci.wedontsleep.org/job/db-devel/784/
<wgrant> wallyworld: Did you have to configure a custom cwd somewhere to get bin/run debuggable>
<wallyworld> wgrant: you set the wd in the debug config
<wallyworld> plus you need a custom PYTHONPATH variable: parts/scripts:lib:$PYTHONPATH
<wallyworld> set this in the same config dialog
<wgrant> Where's the debug config?
<wgrant> The last IDE I used seriously was VS2003, so I'm a little rusty on how things work these days.
<wallyworld> there's a drop down next to the run/debug icons in the top toolbar
<wallyworld> you need to initially set up a debug profile
<wallyworld> for lp. i alsi have one for runnign tests
<wgrant> Ahh, thanks.
<wallyworld> the script is bin/test
<wallyworld> the params are -vvct foo
<wallyworld> for running, the script is bin/run
<wallyworld> the params are -r librarian,google-webservice,memcached -i development
<wgrant> lifeless: You has qa.
<lifeless> hah
<lifeless> parallel - 15 minutes
<lifeless> single threaded 13 minutes
<lifeless> Total: 1338 tests, 0 failures, 0 errors in 12 minutes 38.127 seconds.
<lifeless> id=4, tests=1686, failures=2, skips=29
<wgrant> Heh.
<lifeless> the extra tests are layers - different reporting
<spm> parallel is slower? that sounds like doing it wrong. :-)
<lifeless> yes
<lifeless> this is why 'in dev' not 'done'.
<spm> pfft. details. and facts. they have no place here.
<lifeless> aka 'dont bother me with facts'
<wgrant> lifeless: Last staging fastdowntime was offline for 91s.
<wgrant> Looks like my optimisations last night were effective.
<lifeless> getting there
<lifeless> wgrant: thanks for digging in!
<lifeless> thats still running trusted.sql ?
<wgrant> Will hopefully dig harder to reduce sync overhead tonight.
<wgrant> Yes.
<lifeless> be good once we eliminate that
<wgrant> Overhead for running that too is now like 2s.
<wgrant> So not that bad.
<lifeless> I have to pop out and buy a baby capsulte
<wgrant> Uhoh.
<lifeless> wgrant: thats still 2.5% :P
<lifeless> wgrant: I will poke at qa upon my return
<wgrant> Hmm, we reset permissions serially :(
<lifeless> wgrant: but it all should be super shallow
<wgrant> That will save 30s on production, but only 5s on staging.
<wgrant> lifeless: Thanks. We're OK to deploy otherwise.
<StevenK> lifeless: Attempting to create an Oneiric EMI
<lifeless> StevenK: cool
 * wgrant curses postgres a bit.
<lifeless> oh?
<wgrant> Tried to parallelise security.py
<wgrant> But postgres does *not* like concurrent group modifications.
<lifeless> will block
<wgrant> Nope.
<wgrant> Get lovely "tuple concurrently updated" exceptions.
<lifeless> WIN
<lifeless> WIN WIN WIN WIN WIN
<wgrant> Yes.
<wgrant> So I try to guess the hosts from the connstrings, and run a separate set of threads first to reset the roles once per host.
<wgrant> Seems to work.
<wgrant> But ew.
<wgrant> I wonder if it gets faster if I share the PermissionGatherers.
<wgrant> Overhead is down from 6s per slave to 0.5s, but that's still pretty bad...
<lifeless> what is per slave?
<wgrant> Hm?
<wgrant> Permissions aren't replicable, so must be set separately on each slave.
<lifeless> can't we do them concurrently on each slave?
<wgrant> This presently takes 6s per slave, serially.
<wgrant> Yes.
<wgrant> But you can't just run multiple instances of the script.
<wgrant> 1) slow startup
<wgrant> 2) if there are multiple DBs on one machine (eg. prod standalone + slave, or staging, or dev), ALTER ROLE boom.
<lifeless> erm
<lifeless> do you mean multiple DBs on one pgsql instance ?
<wgrant> Yes.
<lifeless> we could move them to separate clusters, same machine.
<wgrant> Ew.
<jtv> wgrant: since you're working with the PermissionGatherers, a note of caution that I may or may not have preserved as a comment: I tried more aggressive bundling and found that performance got worse, not better.
<jtv> Also, I really wonder whether all the restrictions we put in security.cfg pull their weight.  Say we isolated a few tables that need special SELECT permission, and went back to granting blanket SELECT permissions to the rest.  Less need for detailed testing, fewer pointless privilege errors, but more relevantly for this: fewer security.cfg changes.
<wgrant> Julian has argued similarly.
<wgrant> It's unfortunate that there's no way to really batch permission changes, without poking pg_class.relacl directly.
<wgrant> Which may not be entirely unreasonable...
<StevenK> Just disgusting
<wgrant> Apparently very disgusting.
<StevenK> So disgusting, jtv left in disgust.
<wgrant> Yes.
 * StevenK *might* have a working Oneiric EMI
<StevenK> Well, I'll be able to actually check if the instance state ever moves from pending
<StevenK> Bah, my branch failed with test_feature_info failing
<nigelb> oh oh
<nigelb> someone broke something?
<nigelb> Reported by Matthew Paul Thomas (mpt: 5235) [ubuntu-bugcontrol] on 2005-11-18 and last updated on ": {"date_
<nigelb> the last updated doesn't seem to be working :)
<wgrant> nigelb: That looks like a Greasemonkey script.
<wgrant> We don't show team memberships, karma, last updated.
<wgrant> And we only show usernames to a few people.
<nigelb> wgrant: Oh, that. It started working recently. I'll check with Brian :)
 * wgrant fixes the db-devel breakage.
<wgrant> jtv: So, I wonder if we might be better off querying pg_class.relacl, working out what permissions are currently there, and then applying only the diff.
<wgrant> jtv: Rather than revoking and granting *everything*.
<jtv> I thought someone had already implemented incremental changes.
<wgrant> jtv: There's --no-revoke
<wgrant> Which, as the name suggests, doesn't revoke. Or change owners.
<jtv> It doesn't revoke?  Or it doesn't do the blanket revoke at the beginning?
<jtv> ISTM if you skip the blanket revoke at the beginning, that breaks privilege revocation.
<wgrant> jtv: Right, it breaks revocation.
<wgrant> We run --no-revoke for nodowntime deploys.
<wgrant> And do the revoke only during (fast)downtime.
<jtv> And that's the problem you're trying to solve right now?
<wgrant> jtv: Currently a full security.py run takes 6 seconds per DB.
<wgrant> jtv: And these are run serially.
<wgrant> Production has 7 DBs.
<wgrant> Which means we are going to spending 42 seconds of downtime just resetting permissions.
<wgrant> Which is less than ideal.
<jtv> That reminds me of something else.
<wgrant> Hmmm.
<wgrant> autovacuum causes fastdowntime to abort, it seems...
<jtv> ISTR the performance increase I got from bundling the grants (not related, I'm sure) shifted the performance bottleneck to somewhere else, even within security.py.  That was just for one database though.
<wgrant> lifeless: :(
<lifeless> wgrant: ?
<lifeless> wgrant: My mind reading is pretty good, but not enough to get much from that :)
<wgrant> Profiling slony.
<wgrant> It is slow.
<wgrant> ddlScript_complete_int, in particular.
<wgrant> Which turns replication back on.
<wgrant> Looks like LOCK TABLE blah IN ACCESS EXCLUSIVE MODE takes about 50ms.
<wgrant> So not really slony.
<wgrant> More postgres.
<lifeless> stub is aiming to get u1 onto slony 2
<wgrant> (it does that for every table)
<lifeless> then upgrade us
<wgrant> I wonder if it's any better.
<lifeless> slony 2 does not do that
<wgrant> slony1-2
<wgrant> Nice package name.
<wgrant> Huh, it renames the relations?
<wgrant> updateRelname confuses me.
<wgrant> Locking takes 45ms, checking for trigger conflicts 5ms... 50ms*250 = 12.5s :/
* wallyworld changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 247 - 0:[#######=]:256
<wallyworld> lifeless: in lib/canonical/configure.zcml, there's an include package="lazr.uri" but there's no configure.zcml in /usr/lib/python2.6/dist-packages/lazr/uri and so it complains. is it mandatory to have a configure.zcml file for such includes?
<lifeless> I suspect skew over time
<lifeless> rem
<wallyworld> you mean delete the include?
<lifeless> no
<lifeless> the package is faulty
<lifeless> we use the egg
<lifeless> ls -l eggs/lazr.uri-1.0.2-py2.6.egg/lazr/uri/configure.zcml
<wallyworld> ah right. thanks. will remove the package
<wgrant> wallyworld: I struck that today in pycharm, then it magically started working again.
<wallyworld> wgrant: yes, it only started happening once i used buildout
<wallyworld> wgrant: you like pycharm?
<wgrant> Mmm.
<wgrant> Ugly and slow.
<wgrant> But maybe.
<wallyworld> slow? really? ugly is a matter of opinion. there's several different themes to choose from
<wgrant> I can't get it to render fonts nicely.
<wallyworld> i use Alloy Bedoin
<wallyworld> and override default fonts by Verdana
<wallyworld> in the Appearance settings
<lifeless> wallyworld: if its competing with vim for perceived performance, it is really quite challenging :)
<wallyworld> there's more to performance than startup time or whatever. things like code navigation, change markers erc rock and save sooo much time
<wgrant> They do.
<wgrant> But I'm used to my editor not having visible latency.
<wgrant> Even PyCharm's menus lag.
<wallyworld> it doesn't for me
<wallyworld> there must be something else happening
<wallyworld> on your system
<wgrant> Hmm
<wallyworld> i think it's fast for bigjools too
<wallyworld> i'd say there's definately an issue somewhere
<wallyworld> maybe with java?
<wallyworld> you using sun's jdk?
<wgrant> I am.
<wallyworld> hmmm. there goes that theory
<wallyworld> wgrant: one thing to do - make sure your pycharm/bin/pycharm.vmproperties has more max memory specified
<wallyworld> -Xms232m
<wallyworld> -Xmx912m
<wallyworld> -XX:MaxPermSize=120m
<wallyworld> -ea
<wallyworld> lp project likes to use a lot
 * wallyworld has to start getting ready for soccer
<wgrant> lifeless: Ahh.
<lifeless> wgrant: and again
<lifeless> wgrant: CONTEXT
<wgrant> lifeless: Slony2 doesn't really disable the triggers... it sets a value which turns them into no-ops.
<wgrant> So it should be really quick.
<wgrant> That makes a bit more sense.
<wgrant> Just to make upgrades fun, slony1 and slony1-2 packages conflict!
<lifeless> yes
<lifeless> two fastdowntimes
<lifeless> or something
<wgrant> One to uninstall slony and move everything to the master, then upgrade slony everywhere, then another fastdowntime to install slony2, then rebuild slaves, then get things talking to the slaves again?
<lifeless> yeah
<mrevell> Morning
<wgrant> 2011-08-05 08:23:57 INFO    Outage complete. 0:00:26.220996
<wgrant> 15s of that was security.py, which can hopefully be optimised down to less than a second.
<wgrant> Morning stub.
<stub> Urgh
<wgrant> That bad?
<stub> My sleep schedule is utterly broken
<stub> again!
<lifeless> 2:30pm ?
<stub> 3:30pm
<wgrant> Ow.
<nigelb> Fixing that is going to be painful.
<jtv> stub: I missed dinner last nightâ¦ ended up staying at home and, from what it looks like, drinking myself into a hangover from Tesco fruit juice.
<jtv> I would recommend it, except it doesn't help us much as I'd hoped.
<jtv> Did you get a chance to snuggle in at the trough?
<jtv> At John's, I mean.
<wgrant> Hmmm.
<wgrant> Interesting.
<wgrant> If I add new slaves without restarting the existing slons, syncing is really slow.
<wgrant> Like, slony1-slow.
<wgrant> But when I restart them all, it drops back to 2s.
<wgrant> And we have 30s outages again.
<wgrant> Oddity.
<wgrant> stub: Does that happen with slony 1 too? I've upgraded to 2 to see how less crap it is.
<wgrant> stub: With slony 2, sync lag around the upgrade is now the slon interval, rather than >20s.
<stub> jtv: If you have a hangover, it might have been past its use by date...
<stub> jtv: Yup, and take away. Good stuff.
<jtv> stub: possiblyâ¦ I think I just drank too much.
<stub> wgrant: Cool. We have other reasons to switch, and I want to install an Ubuntu One system with slony2 if I can get the packages sorted.
<stub> wgrant: You found slony2 packages built somewhere?
<stub> I think they are in Debian, but don't think they have leaked through to Ubuntu yet due to no upgrade path between slony1 & 2.
<LPCIBot> Project db-devel build #785: STILL FAILING in 5 hr 57 min: https://lpci.wedontsleep.org/job/db-devel/785/
<stub> oic - silly version numbers hiding it from me.
<stub> Need to get 2.0.7 still - Natty has 2.0.6
<stub> wgrant: so did the existing scripts all work unmodified?
<wgrant> stub: Sorry, was eating.
<wgrant> stub: Yep, everything works fine.
<wgrant> Just installed the new package (which conflicts with the old one) and rebuilt the cluster from scratch.
<wgrant> No issues with that, new-slave, or full-update.
<wgrant> oneiric still has 2.0.6 :(
<wgrant> sid too.
<wgrant> Sad
<stub> 2.0.7 was only recently released.
<wgrant> stub: With 4 slaves and a multithreaded security.py, 31s outage.
<wgrant> security.py is the main limiting factor now.
<wgrant> It's about 15s with 4 slaves, even multithreaded.
<stub> wgrant: you still seeing a high variance though when slony is bounced/not bounced before the upgrade?
<wgrant> stub: Yes, it adds a good 15-20s.
<wgrant> I thought it was just really bad scaling, but then bounced the slons and all was good.
<stub> security.py could be optimized further separately. eg. run '--no-revoke' before the outage and add a '--revoke-only' argument for application during the outage.
<wgrant> stub: I think it may be better to examine the permissions and owners and just apply a diff.
<wgrant> Since presently it revokes everything then grants everything, so --revoke-only isn't really plausible.
<wgrant> We already inspect pg_class in a few places... not toooo terrible to also interpret its relacl.
<stub> Yer. Haven't poked at that part of the schema - just used GRANT and REVOKE
<wgrant> I'd be reluctant to update relacl manually, but using it to work out a diff seems reasonable and quick.
<stub> We will still need to do revokations though, which is the potentially blocking operation. Grants can be done outside of the outage.
<wgrant> Sure, but if we're only applying a diff it should be really quick.
<wgrant> So we can probably do it all in one hit.
<stub> We already are really quick. We are just changing the definition of really quick :-)
<wgrant> 'SELECT relnamespace, relname, relacl FROM pg_catalog.pg_class;' can't be too slow :)
<wgrant> Then it's just a matter of parsing them into something that we can compare to security.cfg and diffing.
<bigjools> wgrant: hey you used PyCharms buildout support then?
<wgrant> Not a huge amount of code, and should be very quick.
<stub> Depends on how many corresponding relacl rows there are, and if all the role membership stuff is exploded or not.
<wgrant> bigjools: I did. Possibly slightly unreliable (had some odd import issues), but seems to work.
<bigjools> gotta be better than setting all the egg directories
<wgrant> Exactly.
<wgrant> I used bin/i18nextract for paths.
<wgrant> Since only the i18n scripts have them.
<wgrant> stub: It's not exploded.
<stub> Love my weather indicator. Bangkok is expecting Bacon.
<bigjools> !
<wgrant> Heh
<stub> wtf is 'light rain' horizontal wavy lines?
<wgrant> I thought that was fog.
<stub> That would be 'light rain is a really, really strong wind'
<stub> Not in Bangkok obviously
<bigjools> I have my own weather station, it'd be nice if I could make HTC's weather app use it.  But no.
<allenap> Anybody fancy doing a review?
<bigjools> allenap: you are on holiday, bugger off
<allenap> bigjools: Hehe. I'm catching up a big I missed from yesterday :)
<stub> I'll have your holiday if you are not using it. Ta!
<jtv> cjwatson: got a moment to talk about bug 659769?
<_mup_> Bug #659769: should copy custom uploads to newly-initialised release <derivation> <lp-soyuz> <new-release-cycle-process> <soyuz-publish> <Launchpad itself:In Progress by jtv> < https://launchpad.net/bugs/659769 >
<nigelb> stub: You're based out of Bangkok now?
<stub> yes
<nigelb> nice
<stub> Come on over, the weather is bacon!
<nigelb> heh
<nigelb> I'm glad there's someone at timzone between me and .au
<stub> I don't know what timezone I'm in. Not Bangkok's, certainly.
<stub> Probably in the middle of the Atlantic at the moment.
<nigelb> hehe
<nigelb> I've figured out that my brain thinks I live in Iceland.
<lifeless> stub: nah, you're running on france time more or less
<jml> I'm looking at a perl script
<jml> The first line is a comment, '# Load everything in a big hash'
<jml> *every* perl script does that at the beginning
<jelmer> jml: I think the hash at the beginning of that line is probably not big enough to fit everything. :P
<jml> jelmer: :D
<wgrant> stub: Is the admin role still used?
<jelmer> sigh, I hate eggs.
<wgrant> stub: (it's the only remaining hole in my diffing security.py, as it's difficult to map the ALL that is granted to admin to real permissions)
<stub> wgrant: I don't think the admin role is used
<stub> Maybe for testadmin?
<wgrant> Argh, yes.
<wgrant> I guess I might specialcase it for revocation too.
<stub> We could explode the ALL into real permissions by object type
<wgrant> But just using a grant/revoke diff takes it from 6s to 1.5s per slave... will look at roles next week.
<wgrant> I considered that.
<wgrant> But they vary by type and version, so mrrrh.
<wgrant> ALL is only used for admin, so I'll just special case it for revokes as well.
<stub> We could also just do those grants outside of the outage
<stub> We revoke ALL?
<wgrant> The current security.py revokes ALL from everyone on everything.
<wgrant> Then grants the security.cfg permissions, and gives ALL to admin on any object mentioned in security.cfg.
<stub> ic
<wgrant> Now I interpret relacl, turn the security.cfg + ALL to admin stuff into a dict, and work out differences. Perhaps I could just assume that admin never gets anything revoked.
<wgrant> Ah, but it will also never get anything granted.
<wgrant> So I probably do need to work out what's in ALL for each type.
<wgrant> Any idea where I would find that?
<stub> wgrant: I wonder if 'grant ALL to ADMIN ON foo, bar, baz...' would work fine - one statement.
<wgrant> It does.
<wgrant> I guess I could just do that.
<wgrant> Outside the loop of everything.
<wgrant> Just always do that.
<stub> Yup
<wgrant> That single extra statement can't hurt too much.
<stub> Two statements, but big ones :-)
<stub> Or not revoke - one statement
<wgrant> Well, the thing is that we only revoke from tables and roles that we know about.
<wgrant> And admin has ALL on everything that we know about.
* bac changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: bac | Critical bugs: 247 - 0:[#######=]:256
<stub> So REVOKE seems pointless
<wgrant> So there's nothing to revoke, unless tables have been dropped from security.cfg, in which case we don't know of their existence.
<wgrant> Right.
<wgrant> So I'll just do a mass grant.
<stub> yup.
<stub> http://onefte.com/2011/06/27/youre-such-an-enabler/
<stub> mthaddon: Any progress on firewall rules and pgbouncer on wildcherry?
<stub> ECHAN
<wgrant> That adds almost 200ms to each node :(
<wgrant> Ah, that's better.
<wgrant> Looks like I can save a second by iterating over the objects as a set rather than a dict :S
<wgrant> Ahem, no, I just failed to add everything to the set.
<wgrant> 2011-08-05 12:42:28 INFO    Outage complete. 0:00:20.629842
<wgrant> yay.
<bigjools> howdy bac
<bac> hi there bigjools
<bigjools> bac: you reviewing?  I have a wee branch
<bac> you wee'd on a branch?
<bigjools> frequently.  It's a long trip to the house across the garden
<bac> uh, that's what a garden is for
<bac> i'll have a look at your branch.
<nigelb> eww :P
<bigjools> bac: https://code.launchpad.net/~julian-edwards/launchpad/queue-links/+merge/70554
<bigjools> thanking you
<bac> bigjools: i looked at the link on dogfood but i don't see what you describe.  do i need to do something to trigger it or am i looking at the wrong place?
<bigjools> bac: look down at the package called "adapt"
<bac> ah, gotcha
<bac> bigjools: r=bac with one bikeshed
<bigjools> bac: oooo :)
<bigjools> bac: heh - in soyuz, we bikeshed you
<bigjools> you know about all the acronyms :)
<abentley> henninge: bzr alias mpull="pull -v -d :submit"
<abentley> henninge: bzr alias spump="pump --from-submit"
<sinzui> jcsackett, I have looked at bugs, branches, teams, and P3As. I think bug 298152 is really fixed
<_mup_> Bug #298152: privacy portlet is not visible enough for indicating private objects <disclosure> <lp-web> <oem-services> <privacy> <qa-needstesting> <ui> <Launchpad itself:Fix Committed by jcsackett> < https://launchpad.net/bugs/298152 >
<jcsackett> sinzui: fantastic. :-P you beat me to all my qa this morning.
<sinzui> I found a bug that your work should have fixed, and in deed it did. I kept qaing to be certain
<jcsackett> cool.
<abentley> bac: Could you please review https://code.launchpad.net/~abentley/launchpad/induce-latency/+merge/70571 ?
<danilos> bac, hi, I've got a 20 line diff on https://code.launchpad.net/~danilo/launchpad/bug-810116/+merge/70564 as well, if you can find the time to review it, that'd be great
<abentley> bac: Actually, let me resubmit that as a new branch.
<abentley> bac: Could you please review https://code.launchpad.net/~abentley/launchpad/induce-latency/+merge/70571 ?
<abentley> bac: I mean https://code.launchpad.net/~abentley/launchpad/local-latency-port/+merge/70572 ?
<bac> abentley, danilos: queued
<abentley> bac: ty
<bac> danilos: due to the tininess of your branch and the lateness of your hour i jumped you to the top.  r=bac
<abentley> henninge: This is the scripting framework I accidentally wrote: https://code.launchpad.net/~abentley/launchpad/local-latency-port/+merge/70572
<danilos> bac, thanks
<bac> sinzui: are you familiar with ian's person-picker expander work?
<sinzui> I am
<sinzui> oh, I reviewed his branch about 30 minutes go
<sinzui> bac Sorry. I think I stepped on you
<bac> sinzui: so you did.  np.  i couldn't get it to work but i think it is b/c i didn't rebuild js
<bac> i would like to see it in action, though
<sinzui> bac: you would need to do that
<bac> and i am...
<bac> :)
<henninge> abentley: cool
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #786: FIXED in 5 hr 53 min: https://lpci.wedontsleep.org/job/db-devel/786/
<bac> sinzui: i'm still not able to see the expander in action.  i
<bac> i've confirmed i have ian's changes loaded in the launchpad.js that is being used
<sinzui> Do you also have the feature flags enabled for the picker's new features
<bac> gah!!!
<sinzui> bac: you want the disclosure rules on: https://qastaging.launchpad.net/+feature-rules
<bac> no mention of FF in the mp and i didn't look at the entire branch
<bac> thanks
<sinzui> bac: it depends on a lot of other branches. one I QAed just an hour ago
<sinzui> We are moving a little too fast. I am taking a break from pickers today
<bac> sinzui: now i see it.  thanks!
<bac> sinzui: did ian consider using the gallery accordion widget?
<sinzui> bac No.but now that you mention it, it might solve the dead left margin created by the expander
<bac> abentley: your MP has henninge as a reviewer.  is he working on it?
<bac> sinzui: it's already in the tree.  may be overkill, may be a nice solution
<abentley> bac:  I just told henninge about it, but maybe he decided to review it.
<henninge> abentley, bac: I am reviewing it but got distracted
<bac> abentley: ok, it looks like he claimed it then.  i'll not do it unless asked
<bac> henninge: ok.  let me know if you need to punt it back
<henninge> abentley: wrong channel? ;-)
<abentley> henninge: yeah.
<henninge> abentley: I believe so, yes. Every text field will be obfuscated.
<henninge> which reminds me we still need special fields for revisions
<abentley> henninge: I think we should call the type "identifier".
<henninge> fbm
<henninge> abentley: review done, needs fixing.
<cr3> benji: if I have an interface exporting a List(value_type=TextLine()), how can I require that the list not be empty?
<benji> cr3: "exporting" meaning over the web service API?
<cr3> benji: yep, I meant as part of @operation_parameters
<benji> cr3: ah; I am not aware of a way to do that declaratively, you'll need to raise an exception if that precondition is not met
<danilos> stub, btw, does not ALTER INDEX RENAME TO not work with primary keys? (for table renames and slony)
<cr3> benji: might you happen to know offhand which exception I should raise that would be the same as not providing an argument for other types?
<benji> cr3: Python raises a TypeError if not enough arguments are provided, I don't know offhand if lazr.restful does something different before Python gets a chance
<cr3> benji: I'll try it out, thanks!
<benji> np
<abentley> henninge: I've updated the docs and pushed.  I don't understand your implementation suggestion.  Could you explain it?
<henninge> abentley: sure
<henninge> abentley: I was thinking along the lines of this: http://paste.ubuntu.com/659367/
<henninge> abentley: or maybe I got that wrong?
<henninge> abentley: I assumed defaults can be specified in this manner.
<abentley> henninge: They can be.  Doing it as a separate step is a bit of a relic of a previous state of the code.
<henninge> I find it a bit strange that getargspec returns None if no defaults are given.
<henninge> It should just return an array of Nones
<henninge> nuns?  ;-)
<henninge> it's Friday
<abentley> henninge: Yeah, me too.  Hate code that treats None and empty container as the same.
<abentley> henninge: Anyhow, updated and pushed.  Is that what you meant?
<henninge> oh, diffs are agregated
<henninge> abentley: I just realized I had missed two revisions
<henninge> abentley: I had not seen the type being inferred from the default (13598)
<henninge> failed to reload
<abentley> henninge: Ah.  I did it while brad was working on the other review.
<henninge> abentley: I should have been clearer about reviewing it.
<henninge> but since I was looking at it, I thought I might as well review it ...
<abentley> henninge: It was a nice thing to do.  Oh well.
<henninge> abentley: one more thing that I forgot
<henninge> abentley: getattr(function, '_helps', {}) can be cached, too, I believe
<henninge> arg_helps
<abentley> henninge: sure.  will do.
<henninge> abentley: but anyhow, this *is* what I had in mind.
<henninge> abentley: what about a test? Or do you think this can do without?
<abentley> henninge: We don't usually test our utilities, do we?
<henninge> ah, right.
<henninge> abentley: yes, I had just arrived at that conclusion, too ... ;-)
<henninge> abentley: r=me, thanks for the good work
<henninge> actually, I'll wait until you pushed that last change
<abentley> henninge: pushed.
<henninge> abentley: approved. See you on Monday.
<abentley> henninge: ta!
<stub> danilos: No. Slony caches the name of the primary key in one of its internal tables and isn't smart enough to notice when it has changed.
<stub> danilos: I think they stopped doing that in the 2.0 series, or it could be fixed in 1.2 if anyone could be bothered.
<benji> bac: if you would, please add https://code.launchpad.net/~benji/launchpad/bug-590014/+merge/70614 to your review queue; thanks
<bac> benji: ok
<bac> benji: did you move the template files from being registered views to properties on the existing view necessary to take advantage of the cachedproperties?
<benji> bac: partly, also so the view wouldn't be instantiated more than once (duplicating queries the second and third times it was instantiated)
<bac> benji: right.  smrt.
<bac> benji: do you have any demonstration that it actually worked?
<benji> bac: heh, yep; I used the ++profile++ view on a dev instance to track the queries that were actually executed before and after
<bac> benji: ok.  it looked like it *should* be helpful
<benji> :)
<jcsackett> sinzui: did you see that bug 89476 was marked fixed release on lp? is there an associated bit you want to deal with or shall i just delete the card?
<_mup_> Bug #89476: busted permissions: cannot unsubscribe ubuntu-security when private <disclosure> <lp-bugs> <Launchpad itself:Fix Released> < https://launchpad.net/bugs/89476 >
<sinzui> jcsackett, I see lifeless marked it as fix released
<sinzui> He saw my tag I think
<jcsackett> dig. i have kanban open, i'll kill the card.
<sinzui> jcsackett, mumble?
<jcsackett> sinzui: sure, one moment.
<abentley> bac: could you please review https://code.launchpad.net/~abentley/launchpad/resubmit-target-prerequisite-same/+merge/70615 ?
<bac> abentley: ok
<dobey> hmm. i keep getting a 500 error when requesting build of one recipe through the API, at least for our bot user. wonder why :-/
<LPCIBot> Project db-devel build #787: FAILURE in 5 hr 51 min: https://lpci.wedontsleep.org/job/db-devel/787/
<sinzui> gary_poster, ping
<gary_poster> sinzui: pong
<sinzui> gary_poster, help.launchpad.net does not know about +structural-subscriptions. I cannot see a link to that page. How do users find it to manage their subscriptions.
<gary_poster> sinzui, that page does not exist.  It is, if anything, an expert interface; something that we might have removed
<sinzui> It certain does exist. I use ever several time a week to manage all mine and your subscriptions
<sinzui> https://bugs.launchpad.net/~gary/+structural-subscriptions
<gary_poster> sinzui, it is intentionally not part of our advertised UI.  It may go away without warning.
<sinzui> okay
<bac> gary_poster: that is the page that gavin put together, right?  it was never completed and our new work make it obsolete.
<gary_poster> bac, correct on the first part.  I agree with your second part, but perhaps that is more arguable (and I have no interest in arguing it).
<gary_poster> well, "never complete" is probably fairly factual as well
<bac> gary_poster: yes, arguable.  but it was purposefully never linked to...a way to make it visible before feature flags
<gary_poster> precisely bac
<bac> my point being, it was never fleshed out and we didn't *remove* an existing link
<gary_poster> right
<sinzui> I do not believe that page is obsolete because it partially addresses a very old issue. Users want a single place to see all their subscriptions
<sinzui> The new work is not hooked into the the user/team bug listing. But I can use the new listing to control multiple subscriptions. I also delete a lot from ~registry
<gary_poster> I hear you sinzui.  However, it was never worked to be considered release-worthy as is, and not deemed important enough by the mgmt to be completed.  We did not remove it in part because you have repeatedly mentioned that you use it.
<gary_poster> However, in terms of "release features when they are done," this was not done.  If I had had more balls, I suppose I would have ripped it out despite everything, because that would have fit in with the decisions around me.
<gary_poster> I did not, and it is no now longer part of my domain in particular--the feature is done.
<gary_poster> our work on it, at least.
<sinzui> understood. I am just addressing user (bug supervisor) concerns that they receive unwanted email
<gary_poster> I would have expected that they could manage that via the link in their emails, sinzui (to the equivalent of https://bugs.launchpad.net/launchpad/+bug/240067/+subscriptions).  Expanding "Other subscriptions" should show everything pertinent.
<_mup_> Bug #240067: Launchpad projects need wikis <feature> <lp-foundations> <ubuntu-platform> <Launchpad itself:In Progress by xaav> < https://launchpad.net/bugs/240067 >
<gary_poster> And let them manage it.
<gary_poster> But maybe there's something special about this case that makes that unusable for some reason
<sinzui> gary_poster, when you get hundred of emails, you do not want a research project to end them. The angry user wants a summary of what went wrong and the power to fix it
<gary_poster> sinzui, yes.  The intent was that you click on a link in an email you are angry about, and fix it.
<LPCIBot> Project devel build #952: FAILURE in 5 hr 59 min: https://lpci.wedontsleep.org/job/devel/952/
#launchpad-dev 2011-08-06
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #788: FIXED in 5 hr 48 min: https://lpci.wedontsleep.org/job/db-devel/788/
<LPCIBot> Project devel build #953: STILL FAILING in 6 hr 13 min: https://lpci.wedontsleep.org/job/devel/953/
<LPCIBot> Project devel build #954: STILL FAILING in 6 hr 0 min: https://lpci.wedontsleep.org/job/devel/954/
<kb9vqf> Have been running a custom Launchpad instance here for some time, but managed to remove my cron entries that handled binary expiration
<kb9vqf> now deleted packages are not being removed from disk
<kb9vqf> Running ./scripts/process-death-row.py does not remove any files
<kb9vqf> is there something else that needs to be run before process-death-row.py ?
<kb9vqf> Never mind, needed to run with --ppa flag
<kb9vqf> :)
<LPCIBot> Project db-devel build #791: FAILURE in 5 hr 59 min: https://lpci.wedontsleep.org/job/db-devel/791/
#launchpad-dev 2011-08-07
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #955: FIXED in 6 hr 15 min: https://lpci.wedontsleep.org/job/devel/955/
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #792: FIXED in 6 hr 5 min: https://lpci.wedontsleep.org/job/db-devel/792/
<lifeless> wgrant: you may have fixed bug 646277
<_mup_> Bug #646277: Targetting to series should result in conjoined bug task <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/646277 >
<lifeless> but probably not, on futher reading
<spm> fyi, bouncing the buildbot master. we have a night of the living dead thing happening.
<spm> and all looks fine again.
<wallyworld_> wgrant: that issue with pycharm debugging and using buildout - there's a bug in the python-lazr.uri package for natty. i've filed a bug 822469. i get around it for now by copying the missing configure.zcml from the tarball to the package
<_mup_> Bug #822469: Missing configure.zcml <lazr.uri (Ubuntu):New> < https://launchpad.net/bugs/822469 >
<wgrant> wallyworld_: Even so, the buildout paths should come first, AFAIK.
<wallyworld_> wgrant: perhaps. but if the bug gets fixed, it won't matter :-) i may file a pycharm bug too
<wgrant> wallyworld_: It will matter.
<wgrant> What if we use a different version?
<wallyworld_> do we do that?
<wgrant> Frequently.
<wgrant> Perhaps not with lazr.uri, but with lazr.restfulclient and zope.interface and others: yes, we frequently run a different version to the system.
<wallyworld_> ok
#launchpad-dev 2012-07-30
<wgrant> StevenK: Morning
<StevenK> wgrant: O hai
<wgrant> StevenK: Your maintainer mail thingy needs QA
<wgrant> And then we can deploy.
<StevenK> wgrant: Yeah, I was been pondering how best to QA that while I was breakfasting.
<wgrant> It should be pretty simple
<wgrant> As you've basically just deleted code
<StevenK> wgrant: If you file a bug under https://qastaging.launchpad.net/auditor, I shouldn't get a notification about it
<wgrant> StevenK: ~auditor-team still has a structsub
<StevenK> wgrant: Removed
<wgrant> StevenK: https://bugs.qastaging.launchpad.net/auditor/+bug/939498 <- the creation notification should have gone to auditor-team, wallyworld and me; the comment to wallyworld and me.
<_mup_> Bug #939498: rhythmbox-metadata crashed with SIGSEGV in _start() <apport-crash> <i386> <precise> <rhythmbox (Ubuntu):New> < https://launchpad.net/bugs/939498 >
<StevenK> wgrant: Noted. Still waiting for Thunderbird to choke down the staging inbox
<wgrant> StevenK: You've not used wallyworld's script to empty it?
<StevenK> I have, there's still 6800 mails in it
<wallyworld_> did you use the param to set the time period to clear messages? perhaps they are recent ones not cleared by default
<StevenK> Default is 2 days
<wallyworld_> yes, so maybe those messages are < 2 days od
<StevenK> Earliest message is 28/07 10:00
<wallyworld_> which is the 2 day window
<wallyworld_> so you need to specify to clear messages starting at 0 days ago
<StevenK> wgrant: So it took qas long enough for the messages to hit the staging inbox. Three mails for the new bug -- and then one for the comment
<StevenK> I guess wallyworld_'s structsub is close only
<wgrant> Ah, yeah
<wgrant> Makes sense.
<wgrant> Sounds good to me.
<StevenK> wgrant: I can put together a deployment for up to r15702 if you wish.
<wgrant> StevenK: Indeed
<wallyworld_> spm: i made a cake on the weekend - the one you sent me a picture of a couple of weeks ago. it turned out ok https://www.dropbox.com/s/irvxydgjw6eolcp/M1410015.JPG
<wallyworld_> spm: i even saved you a piece https://www.dropbox.com/s/yxdksp1uhz450qf/M1410020.JPG
<wallyworld_> so that should get me a few NDTs without begging
<spm> wallyworld_: 1 spm: 0, so throughly outtrolled....
<wallyworld_> well, this time anyway
<spm> wallyworld_: and was it as good as it looks?
<spm> it's looks terribly rich.
<wallyworld_> spm: i think so. ask bigjools
<wallyworld_> spm:  it is very rich, about 1kg of sugar all up
<spm> bigjools: ^^ your considered opinion?
<spm> !!!!!!!!!!!!!!!!!!!!!!!!
<wallyworld_> 900g maybe
<bigjools> spm: it was so good that I didn't need dinner last night
<spm> wallyworld_: was this bribery from your wife to get a cuddle with some new twins?
<wallyworld_> spm: it was her birthday. it was bribery to get me a "cuddle"
<spm> oic
<lifeless> in other news
<lifeless> pybars appears to have users.
<lifeless> Who would have thought.
<lifeless> check the download count on http://pypi.python.org/pypi/pybars
<wgrant> Users or Google?
<lifeless> I'm fairly sure not google.
<lifeless> I've seen much lower counts for other projects.
<lifeless> e.g. http://pypi.python.org/pypi/oops_datedir_repo - 370
<lifeless> and thats still got users (folk @ Canonical to be sure, but virtualenv users...)
 * StevenK stabs django
<StevenK> Trying to use it with making use of django.test.TestCase is just an exercise in futility
<StevenK> ImportError: Settings cannot be imported, because environment variable DJANGO_SETTINGS_MODULE is undefined.
<StevenK> SIGH
<nigelb> StevenK: Wait, why are you using django?
<StevenK> nigelb: https://launchpad.net/auditor
<nigelb> ooh. Nice.
<nigelb> also, yay django
<nigelb> :P
 * nigelb runs
<adeuring> good morning
* adeuring changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
<lifeless> wgrant: how much faster would you say the new bug search queries are?
<wgrant> lifeless: 42
<lifeless> wgrant: haha
<lifeless> wgrant: really, you have no idea?
<wgrant> Pretty much.
<wgrant> I know they're not slower.
<lifeless> darn
<wgrant> Why?
<lifeless> I like being able to say what we've done to make things faster.
<wgrant> Yeah, certainly
<lifeless> but if its 'no slower', its pretty hard to point at and say 'booyah'
<wgrant> It's hard to measure 'cause of all the different cases, and the fact that the access control mechanism changed at the same time.
<lifeless> I know :(
<wgrant> I was also planning to do StormRangeFactory afterwards, but that didn't happen.
<wgrant> That would make the biggest difference.
<lifeless> doing less work :)
<lifeless> do you know off hand how long say, 'firefox' in Ubuntu takes ?
<lifeless> it used to be 3s or so hot to count IIRC
<wgrant> Yeah, Ubuntu packages are where the biggest win is.
<wgrant> Some of them will be more than 100x faster
 * lifeless tries
<lifeless> so close
<lifeless> 1 â 75 of 111109 results
<lifeless> just two more
<lifeless> and it will be epic
<wgrant> Heh
<wgrant> 55ms for a count of firefox's open bugs
<lifeless> oh, late eval faillll
<wgrant> (could be much faster if we added another set of indices, to restrict to just open bugs. that was another plan alongside SRF)
<lifeless> 762.013ms to count forall bugs, 116 to get the batch
<lifeless> 1327.692ms for the batch for firefox itself, 592.421ms to count.
<wgrant> Er
<lifeless> that looks like ~50% of the times we were seeing this time last year.
<wgrant> What?
<lifeless> wgrant: cold cache.
<wgrant> On which instance?
<lifeless> wgrant: 'firefox' FTI in 'ubuntu' context.
<wgrant> Oh
<wgrant> FTI
<wgrant> You didn't say FTI :)
<lifeless> yeah, I did, I said "'firefox' in Ubuntu"
<wgrant> I thought you meant firefox in Ubuntu
<wgrant> ie. https://bugs.launchpad.net/ubuntu/+source/firefox
<lifeless> yeah, I saw that
<lifeless> I would have quoted consistently for that
<lifeless> e.g. not at all, or both.
<wgrant> So yeah
<wgrant> FTI and count are going to be roughly the same length
<wgrant> Because they're pretty similar
<wgrant> ie. both terrible
<lifeless> we've switched index type already right ?
<wgrant> But it's not as problematic as, say, the default Ubuntu listing
<wgrant> Yes
<wgrant> Where the batch of 75 takes a few ms.
<wgrant> But the count takes a second
<wgrant> Anyway, I need to eat.
<wgrant> bbs
<cjwatson> lifeless: Any opinions on ending the beta period of https://blog.launchpad.net/general/beta-test-asynchronous-ppa-package-copies and rolling that out to everyone?  There was one bug, namely bug 1026976, which I fixed a week ago; and I've had a number of positive reports.
<_mup_> Bug #1026976: Archive:+copy-packages creates job which oopses when async-copying from private to public archive <oops> <qa-ok> <Launchpad itself:Fix Released by cjwatson> < https://launchpad.net/bugs/1026976 >
<lifeless> cjwatson: what failure modes do you think it could have ?
<bigjools> did you keep the synchronous exception for small numbers of packages?
<maxb> The async stuff is pretty poor at giving you feedback when a copy fails
<maxb> I guess I should have bugged, I just assumed it was unfinished for now
<cjwatson> maxb: When did you last check?
<cjwatson> bigjools: I haven't removed it yet, but I'd like to; that check is fairly bogus IMO
<bigjools> it's stabbing in the dark, yeah
<cjwatson> The code would be a lot more maintainable if it only had a single well-maintained path
<lifeless> cjwatson: I'm hugely +1 on that.
<cjwatson> lifeless: Failure to copy at all, I guess; the actual backend code is fairly well-tested
<cjwatson> Or UI problems with presenting failures, but I thought that was fixed
<bigjools> given that the backend forms part of ubuntu series opening :)
<lifeless> cjwatson: sounds reasonable to expand it to me, long as you aren't going on leave tomorrow ;)
<maxb> I seem to have lost the email concerned; but anyway; just displaying a message which says a copy failed, with no additional explanation, and emailing an oops code, isn't very nice
<cjwatson> maxb: It's supposed to give an explanation of why - there's a screenshot example in the blog post above
<cjwatson> E-mailing an OOPS code indeed isn't nice.  I thought I'd made it e-mail an actual error.
<maxb> I didn't get an explanation string in the web ui
<cjwatson> Did you click the expander arrow?
<maxb> yes
<cjwatson> If you know what package you were copying, and when, there's some faint possibility I might be able to investigate
<lifeless> how long ago ?
<maxb> ok, this was 6 days ago, and I've found the email now
<maxb> Launchpad encountered an internal error during the following operation: copying a package.  It was logged with id OOPS-952a0272d05f462844490a94901414e9.  Sorry for the inconvenience.
<maxb> was the full body text
<cjwatson> One moment
<cjwatson> Damn, it's expired
<maxb> 24/07/12 23:52
<cjwatson> Is it possible that this was bug 1023372?
<_mup_> Bug #1023372: Direct-copying an already-published package OOPSes <lp-soyuz> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1023372 >
<maxb> Sounds quite likely actually
<cjwatson> Because that's the one case I know of where the UI error would be entirely uninformative
<cjwatson> The job log has expired from carob too
<wgrant> Yeah
<maxb> Although, I believe I was trying to copy to un-delete a deleted package, i.e. previously published, not already-published
<wgrant> NFI why we have only 5 days of retention there
<bigjools> cjwatson: one of my concerns is that all the PCJs go through a single job runner
<lifeless> possibly noone has removed the cowboy after we gobbled all of neems disk ?
<bigjools> cjwatson: and ubuntu series opening will dos ppa copies
<wgrant> bigjools: No.
<wgrant> bigjools: Initiali[sz]eDistroSeries doesn't use PCJs
<wgrant> It doesn't use the copier at all, in the Ubuntu case.
<cjwatson> Indeed, it uses PCRs instead
<bigjools> I clearly need more sleep
<maxb> OK, I've reconstructed the thing I was copying, it was bzr-upload 1.1.0-2~bazaar1~precise1 to bzr/proposed (and then I just uploaded a new version when the copy failed)
<bigjools> cjwatson: I still have the concern though
<cjwatson> Stuff like Archive.copyPackage (e.g. SRUs, syncs from Debian) is our mass-testing of PCJs
<bigjools> syncs will fuck it, yes
<bigjools> which is what I was thinking of and yet somehow typed wrong
<cjwatson> Syncs from Debian are the worst case I'm aware of; I think I've seen that go up to maybe half an hour of backlog, perhaps an hour
<cjwatson> maxb: If you can set up a new example of it and get me an OOPS code, I'm interested
<wgrant> (IDS doesn't actually use PCRs either -- it uses the cloner directly)
<maxb> I'll see if I can retrace the process
<maxb> But in a few hours, at this point - I need to go to work now
<cjwatson> So production jobs run on, what, ackee and loganberry?
<wgrant> cjwatson: Right.
<wgrant> cjwatson: lp:lp-production-crontabs is the usual reference.
<wgrant> You should be able to see that.
<cjwatson> And process-job-source locks for any given job source so you can't run >1 on the same machine easily.
<wgrant> Yeah, but that's easily fixable, as you've found
<wgrant> IIRC we use a separate runner for the PCJs
<cjwatson> That was more process-job-source-groups
<wgrant> So IDS won't be a problem there.
<cjwatson> If you mean bug 839659
<_mup_> Bug #839659: process-job-source-groups.py will starve itself if run with multiple overlapping schedules <Launchpad itself:Triaged> < https://launchpad.net/bugs/839659 >
<wgrant> If that's the one we filed after opening some series recently, indeed.
<wgrant> It's not.
<wgrant> But it may be the same thing.
<cjwatson> I filed bug 991876 separately.  I'm not sure that it's the same.
<_mup_> Bug #991876: initializedistroseriesjob starved by other jobs <derivation> <new-release-cycle-process> <Launchpad itself:Triaged> < https://launchpad.net/bugs/991876 >
<wgrant> Right
<cjwatson> No, not as such.
<wgrant> They're pretty similar.
<wgrant> But different scripts.
<wgrant> Probably the same fix.
<cjwatson> Anyway, the locking in process-job-source itself: what resource is it actually protecting?
<wgrant> Launchpad maintenance engineers' jobs, mostly.
<wgrant> It's just the normal script boilerplate.
<cjwatson> Well, it does seem at least somewhat real.  JobRunner.runFromSource fetches a list of jobs first and then tries to run them all; if it's racing with another runner it's not clear to me that it will behave sensibly.
<cjwatson> I suppose we could have a separate job source depending on whether you were copying to a distribution or a PPA, a bit like publishing.
<wgrant> cjwatson: It should be fine.
<wgrant> cjwatson: Doesn't it try to acquire the lease and skip if it can't, committing between jobs?
<cjwatson> Oh, true.
<wgrant> Anyway, the solution to non-concurrent code is to make it concurrent and fix stuff that breaks.
<cjwatson> Still, multiple runners won't magically avoid starvation, they'll just make queues clear more quickly.
<wgrant> Yeah
<wgrant> There's also celery
<wgrant> I'm not sure what the queueing options are there
<cjwatson> If the goal is for PPA copies never to be starved by distro copies, the answer is for those not to be in the same queue
<lifeless> is the distro copy done as 20K separate jobs?
<lifeless> Or one 20K long job ?
<lifeless> or something in between ?
<cjwatson> You mean syncs from Debian?
<cjwatson> Something in between.  It uses Archive.copyPackages, which in principle could copy the whole lot at once, but in practice it does a little bit of advance checking for each package so it times out if asked to do too many in a single job; so our client for that bisects the list until it finds a set it can do in one go.  It tends to wind up being 100 or so per job.
<cjwatson> Copying precise to quantal (say), as wgrant observes, is a different system not in the same queue as PPA copies.
<lifeless> well, I mean this specific thing you 're worried about starvation thereof
<lifeless> celery lets us have multiple execution units for a single queue
<cjwatson> Then the answer is that the worst case I'm aware of for that tends to be around 20x100.
<lifeless> so, - meh.
<lifeless> I wouldn't spend any time on starvation for such a small set
<cjwatson> Duration-wise, I can't remember exactly but I think it took an hour or so to clear that.
<lifeless> thats entirely tolerable (not desirable)
<cjwatson> Mm.  There's no publicly-visible indication of queue length anywhere, which doesn't help.
<lifeless> if you have time, enabling concurrent runners (w/celery) would be good, that would allow us to have e.g. 8 of them and have them mostly idle
<cjwatson> As far as I'm aware the celery plumbing is in place, but are there any jobs using celery for real yet?
<lifeless> not on prod AFAIK, keeps failing qa
<cjwatson> I wonder if it would be interesting for the in-progress notification of PPA copies to say how long the queue is, or something.
<wgrant> lifeless, cjwatson: Branch scan jobs have been celery'd on prod for a few weeks
<wgrant> It's some related followup work that I am chain-reverting.
<bigjools> does any of this ever get announced?
<bigjools> I try and keep up, but ...
<wgrant> I only found out by finding that the logs were missing :)
<bigjools> :/
<cjwatson> <cjwatson@sarantium ~/src/canonical/lp-production-crontabs>$ grep -i celery *
<cjwatson> <cjwatson@sarantium ~/src/canonical/lp-production-crontabs>$
 * cjwatson wonders how to tell
<wgrant> cjwatson: celery isn't a cronjob
<wgrant> That's why it's better than cronjobs
<cjwatson> Oh, right
<wgrant> You can see celeryd-job.log on ackee
<wgrant> https://launchpad.net/+feature-rules
<wgrant> jobs.celery.enabled_classesdefault0BranchScanJob
<wgrant> Well that wentw ell.
<cjwatson> Where do the runners log?
<cjwatson> Oh, sorry, read up
<bigjools> night all
<cjwatson> Is it likely to be deliberate that PlainPackageCopyJob.createMultiple doesn't call celeryRunOnCommit for each of the new jobs?
<wgrant> adeuring: ^^
<adeuring> cjwatson, wgrant: I don't know -- abentley workod on that part
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: benji | Firefighting: - | Critical bugs: 4.0*10^2
<jam> cjwatson: did you ever get a chance to look at the Maverick filenames? Should I try to regenerate the list, or are you not going to be able to get to it for a while?
<cjwatson> If you can generate something fresh, I can probably look this week, crappy Internet connection chaos notwithstanding
<cjwatson> (My home internet connection has been down for six days)
<deryck> Morning, all.
<rick_h_> morning
<deryck> rick_h_, ping for standup.
<jam> dpm: just a quick ping about where translations is at, did you get a chance to look at the imports
<rick_h_> deryck: ah sorry, didn't get invite
<dpm> hi jam, I just came back from holiday, but some community members manually approved and fixed the imports in the meantime. Once I've finished catching up, we'll open translations. I need to sync up with the other translations coordinators, but I think we can open and announce tomorrow or wednesday
<jam> dpm: sounds good to me
<jam> just checking in on it
<jam> I have no personal time frame in mind :)
<dpm> ok, thanks :)
<sinzui> mrevell: do you have time today to talk about Sharing?
<mrevell> sinzui, I sure do. My two meetings this afternoon have been cancelled, so I'm wide open.
<sinzui> lets talk in 25 minutes.
<sinzui> ^ mrevell
<mrevell> sinzui, ack, speak then
<rick_h_droid> deryck sorry machine completely hung
<deryck> rick_h_droid, np.  We eneded standup.  Will ping you shortly to chat more about mockups.
<rick_h_droid> ok
<deryck> rick_h_ or rick_h_droid -- are you back up?
<rick_h_> deryck: rgr
<deryck> rick_h_, ok, let's chat in 5 then.  And bottom of hour.
<rick_h_> deryck: k
<deryck> rick_h_, let's use the standup hangout.
<timrc> We've been having intermittent build failures due to connection reliability issues with LP... has anyone else been experiencing this problem? Was there a disruption around was their a service planned (or otherwise) disruption around 2012-07-30 01:16:00 UTC ?
<timrc> blah at my awesome editing skills
<czajkowski> timrc: we were due to have disruption at 11am UTC today but it didn't go ahead
<rick_h_> deryck: ping, got a sec to hop back on a hangout?
<jam> cjwatson: devpad: ~jameinel/cleanup-librarian/maverick-2012-07-30.log.gz
<jam> is a new list of files
<cjwatson> Anyone want to comment on my suggested approach to bug 745799?
<_mup_> Bug #745799: DistroSeries:+queue Timeout accepting packages (bug structural subscriptions) <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/745799 >
<rick_h_> deryck: howdy
<deryck> hey hey, rick_h_
<rick_h_> deryck: got a sec?
<deryck> rick_h_, oh, oops.  You asked while I was on another call and forgot to ping back.
<deryck> rick_h_, sorry, dude.  let me grab headset and we can go now.
<rick_h_> yea, np. Know it's busy but need to ask a few ?
<rick_h_> ty much
<abentley> rick_h_: btw, it looks like we will prioritize private blueprints over private questions.
<rick_h_> k
<deryck> rick_h_, I think you is froze again :)
<rick_h_> sinzui: ping, have time for a 2min sanity check hangout?
<sinzui> rick_h_, yes
<sinzui> rick_h_ https://qastaging.launchpad.net/launchpad/+admin
<sinzui> rick_h_ http://www.youtube.com/watch?v=Vq6VsKuFY_o
<rick_h_> sinzui: ty much
<abentley> benji: Could you please review https://code.launchpad.net/~abentley/launchpad/longer-mq-timeout/+merge/117324 ?
<benji> abentley: sure
<benji> abentley: done with your branch
<abentley> benji: ty
<benji> abentley: my pleasure
<sinzui> benji: do you have time to review https://code.launchpad.net/~sinzui/launchpad/register-project-maintainer/+merge/117335
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
<maxb> sinzui: In http://blog.launchpad.net/general/project-maintainer-can-see-private-bugs, people can lose access, not loose it :-)
<sinzui> maxb: alas. I have lost it
<sinzui> I will try to correct the blog
<wgrant> abentley: Hi, https://bugs.launchpad.net/launchpad/+bug/1018235 needs QA -- can you do it or explain what needs testing?
<_mup_> Bug #1018235: TestRunMissingJobs.test_find_missing_ready is disabled due to spurious failures <qa-needstesting> <spurious-test-failure> <test-system> <Launchpad itself:Fix Committed by abentley> < https://launchpad.net/bugs/1018235 >
<lifeless> wgrant: where is the sso decoupling at?
<wgrant> lifeless: Well, I shelved it while waiting for them to land my first branch
<wgrant> lifeless: It took 3 months
<wgrant> And then I forgot to pick it up again
<wgrant> Maybe I should.
<lifeless> there is an RT at the moment around sso staging w teams not there etc etc
<lifeless> it would help, I think :)
 * StevenK stabs mumble
<lifeless> sinzui: how goes the blockage?
<StevenK> wgrant: So I do think test_information_type_does_not_leak is a bong test. It structurally subscribes the product owner, creates a userdata bug for that product and then asserts they aren't notified.
#launchpad-dev 2012-07-31
<wgrant> StevenK: Well, it's completely valid.
<wgrant> StevenK: You're just invalidating the assumptions it's based on.
<wgrant> StevenK: Is that one of the tests that I said you should keep?
<wgrant> StevenK: The key is that not all structsubs will produce notifications.
<wgrant> The one tested in that test does, because it uses the owner for convenience.
<wgrant> So change the test to not subscribe the owner, and it will hopefully pass.
<wgrant> lp.bugs.model.tests.test_bug.TestBugPrivateAndSecurityRelatedUpdatesPrivateProject.test_transition_to_private_grants_subscribers_access
<wgrant> What a test name.
<StevenK> wgrant: So if I change the test to structually subscribe a different person, then it is pretty much lp.bugs.tests.test_bug_notification_recipients.TestBugNotificationRecipients.test_private_bug_with_structural_subscriber
<wgrant> "This seems like a reasonable integration test to retain. You've added a test that sharing => notification in test_private_bug_with_structural_subscriber_with_access, but unless I've missed one this is the only one that tests !sharing => !notification."
<wgrant> I missed one :)
<StevenK> Right
<StevenK> wgrant: That doesn't happen often. :-)
<wgrant> Hah
<wgrant> You should always doubt me.
<wgrant> Unless I'm right.
<StevenK> wgrant: I've addressed all of your comments, aside from the two related to (368-369, 392-396)
<StevenK> Those two can wait until after breakfast.
<wgrant> StevenK: What breaks when you filter direct subscribers?
<wgrant> That bit is not negotiable.
<wgrant> 392-396 might be
<StevenK> wgrant: No fair engaging me with questions when my stomach is threatning to break out and find breakfast on its own.
<wgrant> StevenK: that sounds beneficial.
<wgrant> It means you can keep working on this branch while your stomach eats.
<StevenK> wgrant: Hah
<StevenK> wgrant: So I think your comment of "The function didn't previously return nothing for private bugs, ..." is wrong, since I think it was never actually called for private bugs.
<wgrant> StevenK: It may not have been.
<wgrant> StevenK: But you'll need to check other callsites.
<StevenK> I'm about to filter direct subscriptions so I can show you the breakage.
<wgrant> Great.
<StevenK> wgrant: http://pastebin.ubuntu.com/1120572/ is the crux of the breakage
<wgrant> StevenK: Why does that happen?
<StevenK> wgrant: Like I said on the call, I didn't investigate.
<StevenK> I figured out it works if I don't filter direct subscriptions and left it at that.
<wgrant> Right, you should figure that out. It's probably because it adds grants for direct_subscribers
<wgrant> Depending on what else calls direct_subscribers, you may not want to filter direct_subscribers itself
<wgrant> But a wrapper
<StevenK> wgrant: direct_subscribers is a wrapper around direct_subscriptions
<wgrant> Sure
<wgrant> But it's a wrapper for a different purpose
<StevenK> Hm, nothing seems to call it directly
<wgrant>             # Grant the subscriber access if they can't see the bug.
<wgrant>             subscribers = self.getDirectSubscribers()
<wgrant> That might be relevant
<StevenK> Yes, I was about to dig into callsites of getDirectSubscribers()
<StevenK> Oh, is that working around a bug in the sharing service?
<wgrant> Probably not. Why?
<StevenK> I was guessing self.getDirectSubscribers() == []
<StevenK> Looks like I'm wrong
<wgrant> The sharing service doesn't know about subscriptions. That's not its job.
<wgrant> Now
<wgrant> Filtering Bug.direct_subscribers directly is probably wrong
<wgrant> Since people are still subscribed
<wgrant> They just can't see it.
<StevenK> wgrant: Oh? Shall I substract it from also_notified_subscribers then?
<wgrant> StevenK: They need to be notified if they have access
<wgrant> Like also_notified_subscribers
<wgrant> also_notified_subscribers is only used for notification purposes
<wgrant> It is only meaningful for notification purposes.
<wgrant> So it is sensible to filter that there.
<wgrant> But direct_subscribers isn't just used for notification.
<StevenK> Right, so you *do* want me to filter it in also_notified_subscribers
<wgrant> Not subtract it from :)
<StevenK> Just trying to work out how to filter ...
<wgrant> StevenK: In fact
<wgrant> Hm
<StevenK> self.direct_subscribers_at_all_levels is not the place, since that backs directly onto direct_subscriptions
<wgrant> so
<wgrant> direct_subscribers is just used for exclusion, it seems
<wgrant> I misread.
<wgrant> The real thing you need to filter is the getDirectSubscribers call in getBugNotificationRecipients
<StevenK> Hmmm
<StevenK> Which is in the guts of IBug itself, the rest of my playing has been in BugSubscriptionInfo
<StevenK> wgrant: So not getDirectSubscribers() itself? Since we already figured out that fiddling with recipients after the fact is Bad.
<wgrant> In related news, methods that mutate their arguments are bad.
<StevenK> Who woulda thunk it
<wgrant> StevenK: transitionToInformationType and a couple of other places use getDirectSubscribers for non-notification purposes.
<StevenK> Right, but trying to remove things from BugNotificationRecipients afterwards is what led to a rollback and this branch in the first place.
<wgrant> Certainly.
<wgrant> I didn't suggest filtering afterwards :)
<wgrant> I just implied that you can't do it directly and unconditionally in getDirectSubscribers
<StevenK> wgrant: Modulo some clean up, http://pastebin.ubuntu.com/1120614/
<wgrant> StevenK: Right, that's probably reasonably sensible.
<StevenK> Now to make it work
 * StevenK tries to remember how set operations work
<StevenK> wgrant: I can't just set direct_subscribers, since it expects to be a BugSubscriptionSet, I'm trying to remember the set operation for "Remove everything in set A that isn't in set B"
<wallyworld_> sinzui: i am thinking that it would be better to add a remove icon next to the bug dupe link in the portlet (like is done for removing subscribers on the same page) rather than add a new link to the picker. that way the portlets on the page behave consistently
<wgrant> wallyworld_: It's a bit different, though
<StevenK> Which is intersection(), I thinjk
<wgrant> wallyworld_: Since with the dupe thing you can only have one, and you can change it.
<wgrant> wallyworld_: It's more similar to assignee that subscriber.
<wgrant> s/that/than/
<wallyworld_> which i think is silly also to have to open a picker to remove
<wallyworld_> there should be a remove icon next to the pencil icon for assignees etc
<wgrant> StevenK: >>> s = {1, 2, 3}
<wgrant> >>> s.difference_update({1, 3})
<wgrant> >>> s
<wgrant> set([2])
<wgrant> wallyworld_: Maybe, yeah
<wgrant> wallyworld_: But we probably want to be consistent.
<StevenK> wgrant: intersection_update(), since I want the intersection of the two sets.
<wallyworld_> yeah, i can't decide
<wgrant> StevenK: Oh, bah, missed "isn't"
<StevenK> But BugSubscriptionSet doesn't implement that.
<wgrant> You might have to get to it before it objectifies it.
<wgrant> Or alter getDirectSubscribers' other callsites to not call getDirectSubscribers
<StevenK> No, I need to recreate it, BugSubscriptionSet is a frozenset
<StevenK> ... somehow
<wgrant> Due to @freeze(BugSubscriptionSet), porbably.
<StevenK> The following table lists operations available for set that do not apply to immutable instances of frozenset:
<StevenK> Which intersection_update() is in
<wgrant> Sure
<wgrant> A frozenset is frozen
<wgrant> So updating it doesn't make an awfully large amount of sense.
<StevenK> Yes
<StevenK> direct_subscribers = BugSubscriberSet(
<StevenK>                 direct_subscribers.intersection(filtered_subscribers))
<wgrant> wallyworld_: In r15569 you changed bug filing to not respect extra_data.private if the user is a bug supervisor. Do you recall why?
<wgrant> (it probably means that apport-filed bugs for ~ubuntu-bugcontrol end up public unless someone's watching carefully)
<wallyworld_> hmmm. not of the top of my head. i'll see if i can remember
<StevenK> HAHAHA
 * StevenK notices a typo in accesspolicy
<wgrant> StevenK: Where?
<StevenK>      def revokeByArtifact(cls, artifacts, grantees=None):
<StevenK> -        """See `IAccessPolicyGrantSource`."""
<StevenK> +        """See `IAccessArtifactGrantSource`."""
<StevenK>          cls.findByArtifact(artifacts, grantees).remove()
<wgrant> Copy-paste errors in docstrings of boilerplate methods. How novel :)
<wallyworld_> wgrant: i can
<wallyworld_> can't recall whyt thaT CHANGE WAS MADE
<wallyworld_> arg, caplock fail
<wgrant> wallyworld_: Yeah, I can't see a good reason. I'm replacing that code, so I'll ignore it :)
<wgrant> Thanks.
<wallyworld_> ok, thanks
<wallyworld_> too bad we didn't have tests that failed
<StevenK> wgrant: Diff updated, can you cast your eye over it again?
<wgrant> StevenK: Sec
<wgrant> wallyworld_: Why's +filebug's information type widget part of bugtarget-filebug-guidelines.pt, rather than with the rest of the form?
<wgrant> I don't really understand why that view is separate at all. I wonder if it might be because of the way ProjectGroup:+filebug used to work
<wgrant> Which means it can probably be merged.
<wallyworld_> from memory that's how it was originally all set up
<wallyworld_> that's where the old privacy/security checkboxes were
<wgrant> Huh
<wgrant> Indeed.
<wallyworld_> it's all a a bit of a mess
<wgrant> Ah, of course
<wgrant> Because the security_related checkbox had the name of the security contact in it
<wgrant> So it was project-dependent.
<wallyworld_> yes, makes sense
<wgrant> But ProjectGroup:+index just redirects now, so that can all be merged.
 * wgrant merges.
<StevenK> wgrant: How about now?
<StevenK> Since your 'sec' has turned out to be a description of how long it will take you to get distracted, not how long it will take you to look at my MP.
<nigelb> lol
<wgrant> StevenK: r=me
<cr3> is there a way to have the status of a bug in launchpad set to fix released automatically when a package is automatically built from recipe?
<wgrant> cr3: No. You'd have to write a launchpadlib script to do that.
<wgrant> Recipes are normally used for daily builds, which aren't usually "releases"
<cr3> wgrant: that makes sense actually, thanks!
<StevenK> wgrant: I guess that branch doesn't fix bug 901548, but lays the groundwork?
<_mup_> Bug #901548: launchpad does not send emails to the assignee of private bugs <bugs> <disclosure> <email> <privacy> <sharing> <Launchpad itself:In Progress by stevenk> < https://launchpad.net/bugs/901548 >
<wgrant> StevenK: I think it probably fixes it implicitly.
<wgrant> I'm pretty sure I tested that case, in fact.
<wgrant> But that was like 4 days ago
<StevenK> Then I'll link the bug and branch, thanks.
<wgrant> Like
<wgrant> Why wouldn't it fix it?
<wgrant> It seems like it should
<wgrant> Since the assignee is in also_notified_subscribers
<StevenK> That's a point.
<StevenK> stub: Hai! https://code.launchpad.net/~stevenk/launchpad/db-bugsubscriptionfilter-itype/+merge/117003 would love an review.
<stub> StevenK: What are the different information types again?
<lifeless> huey duey and louie
<lifeless> stub: has mthaddon been in touch w.r.t. django session cleaning mark 2 ?
<StevenK> stub: PUBLIC, PUBLICSECURITY, PRIVATESECURITY, USERDATA and PROPRIETARY
<stub> lifeless: Not yet
<StevenK> stub: The idea being that someone can structurally subscribe to all PRIVATESECURITY bugs.
<lifeless> stub: ok, so - we're triggering mass locks in vacuum, theory is its scanning for pages to fill into but the table is so big... it takes ages *and* holds locks while it does the scan.
<lifeless> stub: so the cleanups are on hold; I've suggested we do a bait and switch to a new table
<lifeless> with a background clone of live sessions + a indexed copy of new sessions only during a FDT-like window.
<wgrant> lifeless: That looks like oops-tools success.
<stub> StevenK: Yeah, I was thinking humans would be interested in 'all', and 'security'.
<StevenK> stub: It is following the currently established pattern, see BugSubscriptionFilterStatus and BugSubscriptionFilterImportance
<stub> StevenK: Yeah, model is fine. I'll leave it up with you guys to design the UI.
 * StevenK digs for a bug
<stub> StevenK: My initial reaction is we have a confusing UI if it allows someone to subscribe to just publicsecurity bugs, or just userdata bugs.
<stub> StevenK: But I'm not reviewing that :)
<stub> lifeless: How did you confirm vacuum is taking locks?
<lifeless> stub: lock dependency graph shows it holding locks, strace shows it reading lots of data.
<stub> lifeless: what sort of locks?
<lifeless> I don't recall, I think they were row level (but rows that sessions were trying to use)
<stub> Right, autovacuum is supposed to not affect anything and release locks and back off if other stuff needs them.
<lifeless> it backs off at the deadlock detection interval
<lifeless> it definitely takes locks out :)
<stub> These are 8.4 though...
<StevenK> wgrant: Hmmm, is there a bug for filter by InformationType?
<StevenK> wgrant: I thought there was one, but I'm having trouble finding it
<StevenK> Ah, found it
<StevenK> stub: Can haz stamp on the MP?
<stub> lifeless: Some vague hints that it might be our use of the CURSOR...
<StevenK> lifeless: Why repeated oops tools mails?
<stub> StevenK: Do you really want that 'id' column?
<StevenK> stub: It's a many-to-many relationship, I think it's needed
<wgrant> StevenK: Testing the new report-specific addresses
<wgrant> StevenK: So we don't get U1
<wgrant> StevenK, stub: It's not necessary, but it's probably best to match the others
<wgrant> (which have the ID, for reasons that are not clear)
<StevenK> wgrant: Ah, testing on production. WCPGW.
<stub> StevenK: Oh. Why isn't the (filter, information_type) unique?
<StevenK> stub: Because I was making it match Status and Importance, but that's not the best answer.
<stub> lifeless: yes, the cursor we use to maintain the list of rows to remove can pin blocks. " If, for example, someone leaves a cursor open, the data block to which  the cursor refers is busy (the technical term is "pinned") and can't be  vacuumed until the cursor is closed or moved to a different block"
<stub> StevenK: At the moment, you can have two identical rows. That means your queries are going to need DISTINCTS all over the place to remove the redundant rows when you join with that table.
<stub> StevenK: There is no reason for (filter, information_type) to not be unique, since there is no other actual information in the table (id doesn't count as information)
<StevenK> stub: So, maybe we fix Status and Importance in the same patch
<stub> And if it is unique, we can drop the id column too making (filter, information_type) the primary key.
 * StevenK tries to cook up a query showing if there is anything in BugSubscriptionFilterStatus that is duplicated.
 * StevenK reaches for mawson
<lifeless> stub: yes
<lifeless> stub: so you think its the cursor, and vacuum is a bystander ?
<lifeless> stub: this conflicts with some evidence we have - killing the cleanup process doesn't unstick servers that this happens to.
<lifeless> stub: I don't know why.
<lifeless> bah
<lifeless> StevenK: I don't know why (oops-tools 2nd mail)
<stub> lifeless: Have you killed the backend server processes when you have killed the cleanup process?
<stub> They tend to hang around with postgresql unfortunately.
<lifeless> I think so. Not 100%
<lifeless> stub: what do you think of the proposed approach anyhow ?
<stub> lifeless: oh, proposed approach is fine. Might take some downtime though unless we much around with triggers and stuff.
<stub> lifeless: Saves us having to run a CLUSTER after to remove the miles and miles of empty pages.
<StevenK> stub: So it looks like there are guards in place in the model code that avoids the need for UNIQUE.
<StevenK> stub: (See -ops)
<lifeless> stub: it looks to me like we could get 4-5 seconds downtime
<lifeless> stub: just need an index on the session expiry
<StevenK> wgrant: Did you see https://www.djangoproject.com/weblog/2012/jul/30/security-releases-issued/ ?
<stub> StevenK: Right. So why do we want the data model to allow redundant rows in the table that are totally indistinguishable?
<StevenK> stub: We don't, and it doesn't at the moment, is what I'm saying.
<lifeless> stub: step A) index that concurrently; B) copy live sessions, C) copy live sessions again but only those with an expiry time >> (time of previous query + expiry window); D) downtime starts
<stub> lifeless: Which should be added anyway for regular cleaning runs
<stub> StevenK: The proposed data model does.
<StevenK> stub: You're having a deep discussion with lifeless, prod me when you're done.
<lifeless> stub: E) copy live sessions again as in C; F) switch tables; G) downtime finishes.
<stub> lifeless: Just going downtime & copying the relevant records across is possibly best. Can do a timing.
<stub> But it isn't the underlying issue, which is that people are seeing behavior in autovacuum that other people don't see.
<lifeless> stub: other folk do see it, rarely.
<lifeless> stub: and you may have found a cause :)
<stub> Yes, so fixing the django script to not hold open a cursor might be the best approach.
<stub> That is just a guess though, based on vague comments. Reports of problems are rare, vague and usually for different situations (like the one I'm reading now, to do with TRUNCATE)
<lifeless> stub: so, see the incident report on the wiki for more info
<lifeless> stub: I'd like us to design around the risk, if we can, as extended downtime sucks :)
<stub> lifeless: What risk are we designing around?
<lifeless> stub: that we don't understand the root cause of the problem
<lifeless> stub: e.g. that it really is just vacuum itself, such as the hidden lock documented to be in 8.4
<stub> You can't design around vacuum. Vacuum is an integral and required part of PostgreSQL.
<stub> You can't even turn it off entirely, despite the magic option in postgresql.conf that should only ever be used in extremely rare use cases.
<stub> Vacuum getting involved here would be a symptom, not a problem.
<stub> I suspect that the problem is the cursor I old open for up to an hour, which is like a long transaction in many ways.
<lifeless> stub: we can design around vacuum - e,.g. the bait and switch approach :) That said, its going to be up to you:)
<stub> It is just intuition, but we should probably fix it anyway since we will hopefully not need hour long transactions with pg_dump anyway.
<stub> lifeless: Guess what happens when you fill the new table with data? It gets vacuumed...
<stub> well... analyzed.
<stub> which is autovacuum
<lifeless> stub: yes, but we've no reason to think autovaccuum is intrinsically a problem
<lifeless> stub: the data we have is a) big table with b) lots of deletes and c) lots of writes
<lifeless> I agree we should fix the cursor *too*, because we need to keep the tables trim.
<stub> We also need to run this script daily or more often or we end up in the same situation.
<lifeless> There is also the question of why we're getting millions of sessions
<lifeless> I guess I'm saying its not either-or.
<stub> Yes, I agree.
<stub> It would be nice if we can avoid the risks of swapping a new table into place, and having to write the script to do it.
<StevenK> stub: So if I'm going to change BugSubscriptionInformationType, I'd rather also change BugSubscriptionFilter{Status,Importance} to match. And I don't think I'm up for two primary key swaps.
<wgrant> The tables probably have about $notverymany rows each, so it's easy.
<StevenK> wgrant: Haha. 108 in status and 46 in importance
<wgrant> That's even fewer than I expected.
<wgrant> I thought maybe a few hundred each
<StevenK> That's on DF, so condiment to taste
<StevenK> wgrant: How do I deal with it in the model, though?
<wgrant> StevenK: __storm_primary__
<wgrant> It'll need to be in a separate branch
<wgrant> Which can land in about 5 minutes
<StevenK> wgrant: Right, but I do land that first and then the db patch that will DROP COLUMN?
<wgrant> Right
<wgrant> Land+deploy the app PK change first
<wgrant> Or the app will murder you
 * StevenK sorts that out, since it's easy
<stub> StevenK: We can go with the id column, but unless there is a reason it is better to drop it as it is unnecessary and slower.
<stub> I don't recall if there was a rationalization for it with BugSubscriptionInformationType
<wgrant> At least we're not in too much danger of wraparound :)
<stub> erm... bugsubscriptionfilerstatus, importance
<StevenK> stub: So I've put up a model change for status and importance that will allow us to drop their IDs
<StevenK> Then I'm not sure what needs to happen on the DB side
<stub> ALTER TABLE foo DROP COLUMN id; DROP INDEX foo_filter_status_idx; ALTER TABLE foo ADD CONSTRAINT foo_pkey PRIMARY KEY (filter, status);
<wgrant> Yeah
<wgrant> Can't use USING, since the index is non-unique for no good reason.
<StevenK> Right
<StevenK> And adding the constraint gets us the index too, with unique for good measure.
<wgrant> Right
<wgrant> A primary key is basically just a UNIQUE index without any nullable columns.
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/no-more-id-for-bsfs/+merge/117374
<wgrant> (in postgres)
<stub> StevenK: You're happy with this, or do you have deadlines? I'd like the model fixed, but it seems to have been broken for some time already.
<StevenK> stub: See above :-)
<wgrant> StevenK: You've checked around to see that nothing references them? I suspect it's safe, but there might be something.
<wgrant> Also, you can't spell =
<wgrant> In the first hunk
<StevenK> Oh bloody hell, how did I miss THAT
<StevenK> wgrant: A bzr grep -i 'BugSubscriptionFilterStatus.id' showed up nothing
<wgrant> StevenK: Have you run the bugsubscriptionfilter tests?
<wgrant> Although I guess an ec2 run won't hurt
<wgrant> Since we can't deploy until tomorrow morning anyway
<wgrant> StevenK: Also, it might want filter_id rather than filter. I forget.
<StevenK> Haha
<StevenK>   File "/home/steven/launchpad/lp-sourcedeps/eggs/storm-0.19.0.99_lpwithnodatetime_r406-py2.7-linux-x86_64.egg/storm/info.py", line 95, in <genexpr>
<StevenK>     for attr in storm_primary)
<StevenK> KeyError: 'filter'
<StevenK> Yeah, we so want filter_id
<StevenK> AttributeError: 'BugSubscriptionFilterImportance' object has no attribute 'id'
<StevenK> Hah
<StevenK> self.assertIsNot(None, bug_sub_filter_importance.id)
<wgrant> Heh
<wgrant> That's probably just checking it's been flushed to the store
<StevenK> It then checks the filter, so it can just check that
<wgrant> Ah!
<wgrant> I finally found a test for apport's default private thing.
<wgrant> I think there's only one.
<wgrant> Hidden in a doctest
<StevenK> Hah
<StevenK> wgrant: I'll be pushing the fixes for that branch in a sec
<wgrant> StevenK: Will look
<StevenK> wgrant: Oh, pft. Fully expecting you to get distracted for 45 minutes again. :-P
<wgrant> Nah
<wgrant> I solved my issue
<StevenK> wgrant: Diff updated.
<wgrant> StevenK: Indeed, r=me
<wgrant> Thanks
<StevenK> I guess no-qa is probably okay
<wgrant> Most certainly.
<wgrant> lp-land --no-qa if you're feeling reasonably confident
<StevenK> wgrant: Not like it matters.
<wgrant> Heh
<StevenK> It could go through ec2 twice and still land in time.
<StevenK> stub: https://code.launchpad.net/~stevenk/launchpad/db-drop-id-for-bsns/+merge/117375
<StevenK> stub: And the diff for https://code.launchpad.net/~stevenk/launchpad/db-bugsubscriptionfilter-itype/+merge/117003 has been updated.
<lifeless> stub: so, can you add a card for fixing this sessions issue, and liase with mthaddon on it ?
<stub> lifeless: yes
<czajkowski> lifeless: morning/evening any thoughts on https://bugs.launchpad.net/launchpad/+bug/1022334  https://bugs.launchpad.net/launchpad/+bug/1030584  they both seem rather similar.
<_mup_> Bug #1022334: data is not saved if launchpad has an error <bot-comment> <Apport:New> <Launchpad itself:New> <apport (Ubuntu):New> < https://launchpad.net/bugs/1022334 >
<_mup_> Bug #1030584: Make +filebug timeouts more user-friendly <Launchpad itself:Incomplete> < https://launchpad.net/bugs/1030584 >
<mgz> ...back button works fine provided you have JS disabled
<mgz> and I didn't think the messing around with "no I need to report a new ug" junk broke that
<mgz> was (coincidentally) testing on staging the other day, which nearly always times out first time
<mgz> yup, it doesn't lose your description at least
<lifeless> czajkowski: the second is a confused user, with a real timeout.
<mgz> you need to it 'next' again though which is lame.
<lifeless> so, I don't think we should try to make form resubmital super reliable; thats a browser issue (and IME firefox preserves values just fine)
<lifeless> we should fix the timeouts when submitting the bugs, and those two bugs have different symptoms
<lifeless> one is a slow insert the other is a slow packaging query
<lifeless> there is probably a dupe of that
<czajkowski> hmm ok
<mgz> lifeless: going back to "please describe the bug in a few words" does make it *look* like launchpad has lost your bug
<lifeless> yes, but it hasn't
<lifeless> the thing is
<lifeless> its diminishing returns
<mgz> even though hitting next unhides it
<lifeless> say 1 in a thousand timeout on submission
<czajkowski> mgz: not many people would think to hit next
<mgz> this is general usability though, not just timeout related
<lifeless> should we make dealing with the timeout easier, or fix the timeout ?
<lifeless> mgz: welllll, its a wizard
<lifeless> so that argument goes both ways
<mgz> a one page wizard that resets to the start at the drop of a hat
<mgz> with JS off it's much better
<lifeless> mgz: I'm not sure what your point is.
<mgz> because the 'search for similar bugs' and 'fill in all your details' are actually seperate
<lifeless> I'm clearly not against improvements, but if I had to choose what to work on, the thing they are complaining about (not the timeout itself), isn't what I would do.
<mgz> so, you can safely use back and such like without things breaking
<mgz> it's like the "error when reporting a bug" issue from the other day
<lifeless> yes, and that one is terrible and I think we should do better
<lifeless> because its a common case not an exceptional case.
<mgz> my point is it's basically the same thing, I agree for timeouts specifically fixing the timeout is more relevent
<lifeless> Its similar in a very technical sense
<lifeless> its not at all similar in a use case sense
<lifeless> IMNSHO
<mgz> so, specifically on czajkowski's question,
<mgz> I'd make the first bug a usability one on using back from navigating off the +filebug page (or dupe against an existing bug on that)
<mgz> and make the second one about the timeout that wgrant dug up (or dupe against similar +filebug timeout)
<wgrant> abentley: Hi. Will you be able to QA bug #1018235 today?
<_mup_> Bug #1018235: TestRunMissingJobs.test_find_missing_ready is disabled due to spurious failures <qa-needstesting> <spurious-test-failure> <test-system> <Launchpad itself:Fix Committed by abentley> < https://launchpad.net/bugs/1018235 >
<abentley> wgrant: yes.
<wgrant> Great.
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: rick_h | Firefighting: - | Critical bugs: 4.0*10^2
<benji> jml: I finally have some time to look at buildout for you (I've been crazy busy with interviews for the last week).  I'm looking at bug 1029715 as well as digesting the comments on https://code.launchpad.net/~james-w/pkgme-service-python/buildout/+merge/116980; if there is anything else I can look at, feel free to let me know
<_mup_> Bug #1029715: bootstrap.py fails if there is a system-wide install of the desired buildout version <Buildout:New> < https://launchpad.net/bugs/1029715 >
<jml> benji: thanks :)
<jml> benji: we've just landed buildout for one of our leaf projects, and hope to move up the DAG today or tomorrow,
<jml> benji: will let you know. very much appreciate the help.
<benji> cool
<benji> jml: note that I will be out for about a week starting tomorrow (moving house 1007 kilometers to the south west) so my responsiveness won't be good ;)
<jml> benji: wow
<benji> just one state over ;) goo.gl/maps/odLpT
<jcsackett_> sinzui: have a few minutes to chat?
<czajkowski> jcsackett: feeling better
<czajkowski> benji: free for a quick pm ?
<jcsackett> czajkowski: feeling semi-functional, thanks. :-)
<benji> czajkowski: sure
<abentley> sinzui: I'm adding Specification.information_type, so I'm looking at patch-2209-12-*.  Do you know why the column wasn't set "NOT NULL" on creation?
<sinzui> abentley: we had to use a garbo job to sync existing data
<abentley> sinzui: Ah!  Thanks.  Mine should include a default, then, I assume.
<sinzui> abentley: the db rollout will only permit a schema change...no code will be released with your column add
<abentley> sinzui: I know.  I'm just hacking the schema for now.
<sinzui> abentley:  you can provide a default in the schema, but I think Lp needs an enum policy in the model, so null would be correct.
<abentley> sinzui: I don't understand.  We should start with all existing specs public and then we can update the model to reflect the column afterward, no?
<sinzui> abentley: bug and branch informationtypes are governed by a policy determined by the presence of a commercial subscription and projects rules to ensure information is not accidentally disclosed. I image you want something like http://pastebin.ubuntu.com/1121640/
<sinzui> https://qastaging.launchpad.net/launchpad/+admin
<abentley> sinzui: could be.  I'll need to find out whether the project's "private" setting will control that.
<sinzui> abentley: I am sure they will. We are adding these now because stakeholders see them as an oversight
<czajkowski> sinzui: do you think you should be able to file a bug from bugs.lp.net or should you go to the project page and file it from there, with the exception being Ubuntu ?
<sinzui> czajkowski: no.
<czajkowski> thats what I'd have assumed
<czajkowski> but wondered did I not know about something
<sinzui> The user has to find the project, find that it uses bugs, and know the bug is not already reported
<rick_h_> jcsackett: never has a mp with just a few lines of comments given me such as headache before :P
<jcsackett> that's alarming.
<jcsackett> why the headache?
<jcsackett> rick_h_: ^
<rick_h_> writing up the MP, sec
<rick_h_> because I founde code doing args.callee.magic_crap and I cried for a while :P
<rick_h_> while trying to understand things
<rick_h_> jcsackett: ok, marked needs fixing, but just because I want to peek at it again.
<rick_h_> let me know if my comments don't make sense
<abentley> rick_h_: have you shared that folder with me yet?
<rick_h_> abentley: ah no, sec will do right now
<abentley> rick_h_: It seems not to matter what email address you use.  You can even sign up for u1 in the process of accepting a share.
<rick_h_> abentley: yea, makes sense with SSO
<rick_h_> sometimes people have different accounts for work/home though so didn't get going on it, see G+
<rick_h_> abentley: but added you, abel, huw and also shared out a G+ folder I've started to put some notes into
<rick_h_> still working on merging paper notes, etc but shared out
<rick_h_> I liked that in the June project how we had that one folder shared to go into for things
<abentley> rick_h_: No, I'm saying you can accept the share based completely on the URL.  It doesn't matter what account you use, and you can even create a new account.
<rick_h_> abentley: oic.
<abentley> rick_h_: I'm happy to use either Google Drive&Docs or u1, but using both seems likely to cause confusion.
<rick_h_> abentley: yea, guess lack of a linux client made it hard to upload all the images to GDrive while docs is our official docs
<rick_h_> I think once we get to mockups vs the hacky wireframes the U1 will go away
<rick_h_> and it'll be real html + docs
<rick_h_> but whatever, I'm happy to play along just starting out with this and can move things as required
<abentley> rick_h_: fair enough, if we assume plain files go to u1, we should be okay.
<deryck> abentley, rick_h_ -- I added a few started cards on the board now.  These are sort of known quantities at this point...
<deryck> abentley, rick_h_ -- but I didn't attempt to get anymore detailed than just the start of our work.
<rick_h_> ok
<rick_h_> deryck: got time to chat?
<deryck> I assume we're working toward getting the board accurate for next work as the week goes along.
<deryck> rick_h_, sure.
<rick_h_> deryck: k, joining hangout
<lifeless> o/
<abentley> lifeless: Hi.  If I want to add a column and put an index on that column, do I need two branches?
<lifeless> depends on the size of the table.
<lifeless> If its small (thousands of rows), no.
<abentley> lifeless: I'm thinking of Specification.  So, yes?
<lifeless> If its large > 10's of thousands of rows, yes.
<abentley> lifeless: "45439 blueprints registered in Launchpad"
<lifeless> yah, I think that will be safest as two patches.
<lifeless> one downtime, one applied hot.
<deryck> hi lifeless.  I expect you'll be hearing a lot from us on Orange over the next few weeks. :)
<lifeless> :)
<lifeless> would you like to have a voice call at some point ?
<deryck> lifeless, you know, we probably should.  Maybe late this week, early next.
<deryck> lifeless, give me time to get coordinated and understand as much as I can first.
<lifeless> sure
<abentley> lifeless: https://dev.launchpad.net/Database/LivePatching sez "All DB schema changes are landed on db-devel", but https://dev.launchpad.net/PolicyAndProcess/DatabaseSchemaChangesProcess sez, "For hot patches the target is lp:launchpad, but for cold patches it should be lp:launchpad/db-devel"
<lifeless> abentley: hi
<abentley> lifeless: hi.
<lifeless> abentley: https://dev.launchpad.net/PolicyAndProcess/DatabaseSchemaChangesProcess is authoritative.
<abentley> lifeless: Okay.  But DatabaseSchemaChangesProcess says Database/LivePatching...[is] the authority [on hot-patching].
<lifeless> heh
<lifeless> abentley: devel is where indexes land. See e.g. r15367
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
<wgrant> abentley: You can't set a default when adding the column on a large table, as that causes every row to be rewritten.
<wgrant> abentley: You'll need to, in one patch, create the column nullable and without a default, and then set the default. Then garbo or similar to set it. Then add the index with a live patch.
<lifeless> (you can add the index before the garbo job; you probably will want to to make the garbo job efficient)
<wgrant> True.
<abentley> wgrant: Thanks.  Sigh.
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/refactor-bsf/+merge/117543
<wgrant> StevenK: Only -102? You disappoint me.
<StevenK> wgrant: :-(
 * wallyworld finds it hard to concentrate looking across the desk at bigjools' ugly face :-(
 * bigjools turns up the music to drown out wallyworld's whinging
<StevenK> wallyworld: Given the difference in height, aren't you staring at his chest, rather than his face?
<bigjools> close
<wallyworld> StevenK: i was until i hoped off his lap
<StevenK> wgrant: Should I add the auditor-team structsub back?
#launchpad-dev 2012-08-01
<wgrant> StevenK: Might as well
<StevenK> wallyworld: Can you go to https://bugs.qastaging.launchpad.net/auditor/+subscriptions and edit your subscription to 'are added or changed in any way'
<wallyworld> sure
<wallyworld> StevenK: do i check any boxes?
<StevenK> wallyworld: The default check of 'send mail about comments'
<wallyworld> StevenK: it's not checked by default, so you want me to check it i assume
<StevenK> wallyworld: Yeah
<wallyworld> StevenK: done
<StevenK> wgrant: Hm, it notified you about new private bug, but not me.
<StevenK> I have a personal structural subscription
<StevenK> And an APG
<wgrant> StevenK: Worked out what's going on?
<StevenK> Nope
<StevenK> Still digging
<wgrant> StevenK: What are the APGs for auditor?
<wgrant> https://bugs.qastaging.launchpad.net/auditor/+bug/939498 shows you
<StevenK> wgrant: I was happy that I could assign wallyworld without an error or him being notified
<_mup_> Bug #939498: rhythmbox-metadata crashed with SIGSEGV in _start() <apport-crash> <i386> <precise> <rhythmbox (Ubuntu):New> < https://launchpad.net/bugs/939498 >
<StevenK> wgrant: I've been playing with 939508
<StevenK> wgrant: me with private security all, private all, and you with private all
<wgrant> Right.
<StevenK> db-drop-id-for-bsns => devel              [OK]       (up for 3:08:33) i-61c8231a   FTR
<wgrant> :)
<StevenK> wgrant: So I don't get why I don't get notified :-(
<StevenK> I own two structsubs, one for auditor-team and one personal
<wgrant> StevenK: 939508 says you're directly subscribed at the moment.
<wgrant> The auditor-team one is invalid, since auditor-team doesn't have a grant
<StevenK> Yeah, but send-bug-notifications.log doesn't mention me at all
<StevenK> And I don't want to flip this bug to PRIVATESECURITY because if you drop out of the notification set it will probably end up empty
<wgrant> Are you sure you don't have 939508 muted or something?
<StevenK> I didn't mute it
<wgrant> StevenK: I wonder if the direct sub filtering is buggy
<wgrant> StevenK: Since you have a direct subscription, your structural subscription will be ignored
<StevenK> Oh sigh
<StevenK> It's not mailing because it's my action
<wgrant> Hah
 * StevenK slams his head through his desk a few hundred times
<wgrant> You have that disabled?
<wgrant> Smart :P
<StevenK> I set it on prod when it was announced
 * StevenK goes looking for it
<wgrant> --
<wgrant> You received this bug notification because you are subscribed to
<wgrant> Auditor.
<wgrant> Matching subscriptions: yes please, I like mail
<wgrant> https://bugs.qastaging.launchpad.net/bugs/939487
<_mup_> Bug #939487: (trunk) gstreamer error breaks the pipe (normal engine), and it's not shown in the dialog <Exaile:New> < https://launchpad.net/bugs/939487 >
<wgrant> Which is private
<wgrant> So it works if you don't have that setting, at least :)
<StevenK> Haha
<StevenK> I just ticked it
<StevenK> Hmmm, changing 939508 doesn't seem to have notified me
<wgrant> StevenK: 2012-08-01 02:22:28 INFO    Notifying me@williamgrant.id.au about bug 939508.
<_mup_> Bug #939508: package tcl8.5 (not installed) failed to install/upgrade: trying to overwrite '/usr/bin/tclsh8.5', which is also in package tcl 8.5.3-2 <apport-package> <i386> <natty> <tcl8.5 (Ubuntu):New> < https://launchpad.net/bugs/939508 >
<wgrant> 2012-08-01 02:22:28 INFO    Notifying stevenk@debian.org about bug 939508.
<_mup_> Bug #939508: package tcl8.5 (not installed) failed to install/upgrade: trying to overwrite '/usr/bin/tclsh8.5', which is also in package tcl 8.5.3-2 <apport-package> <i386> <natty> <tcl8.5 (Ubuntu):New> < https://launchpad.net/bugs/939508 >
<StevenK> wgrant: Shall I land 26-2 to db-devel?
<wgrant> StevenK: Please do.
<StevenK> wgrant: Right, so filtering structsubs looks good
<StevenK> Assignee without access works
<StevenK> Assignee with does too
<StevenK> wgrant: Should I wait for structsubs to be deployed, or toss 26-3 at db-devel too?
<wgrant> StevenK: It has no code deps, right?
<wgrant> We can hopefully deploy both tomorrow
<StevenK> No, just adds the table
<wgrant> If we give lifeless enough cake
 * StevenK lands the DB patch
<adeuring> good mornig
* frankban changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: frankban* (rvba) | Firefighting: - | Critical bugs: 4.0*10^2
<lifeless> wgrant: pybars 0.0.2 - 86 downloads in 1.5 days or so.
<lifeless> wgrant: so pfft
<wgrant> Google.
<stub> lifeless: And buildout. And mirrors. http://pypi.python.org/pypi?:action=display&name=pytz&version=2012d has similar numbers, and wtf is using the 3.1 egg, really?
<wgrant> cjwatson: Does auto_approve really want to skip NEW as well?
<wgrant> That seems like an accident waiting to happen.
<cjwatson> I thought so, yes.  The practical example is auto-sync.
<cjwatson> It's a pain to have to go and manually accept things there, for more or less the same reason that it's a pain to manually accept the results of sru-release.  And it will impede cronning that.
<wgrant> Sure, but default overrides still aren't precisely sensible.
<cjwatson> In practice they're fine for source copies.
<cjwatson> I've not ever noticed them being wrong for auto-sync ...
<wgrant> Indeed.
<wgrant> We just have to hope that nobody ever uses it for binaries without thinking.
<cjwatson> Even then, it'll show up in component-mismatches - it wouldn't be the end of the world.
<cjwatson> The only case where it'd be a problem would be when copying binaries into a stable release from another archive (principally, the kernel).
<cjwatson> But that's already a problem today.
<cjwatson> We change-override them after the fact for now.
<cjwatson> I've not figured out how to override binaries in an include_binaries=True PCJ until after it's published anyway ...
<lifeless> stub: buildout indicates users.
<lifeless> stub: the mirrors use rsync, I thought.
<stub> I don't know
<lifeless> my cats are crazy
<stub> Just know I get 87 downloads in a short time for an egg it is really unlikely anyone would be using. Can't all be mistakes.
<lifeless> I currently have one perched like a pirate bird on my shoulder.
<stub> We haven't seen rentacat around for a while.
<wgrant> cjwatson: Not going to kill the other two in the same branch?
<czajkowski> ohh we have fast internet today BT fixed the line !
<cjwatson> wgrant: Oh, I was doing a branch for each (particularly just in case somebody noticed that pgrestore was in use after all)
<cjwatson> wgrant: I can lob them all into a single branch if you prefer
<lifeless> cjwatson: laziness is a virtue ;)
<cjwatson> I'll tell my manager that at my next appraisal
<wgrant> cjwatson: Anything that's still using it is stupid, and it would be to our significant benefit to break it now.
<wgrant> Thanks.
<cjwatson> wgrant: OK, updated
<wgrant> Way ahead of you :)
<cjwatson> Ow, you're fast
<cjwatson> I was still editing the change description
<wgrant> Descriptions are for people who can't read diffs!
<StevenK> cjwatson: I find it's usually not very good to point out things that can die to wgrant. Because he will usually reply by landing a change that kills it.
<stub> cjwatson: The only place it might be in use is database/replication/Makefile. If it isn't in there, claim your LOC credits.
<cjwatson> stub: It's not.
<cjwatson> That uses pg_restore directly.
<stub> everything else uses pg_restore directly or maybe the duplicate of pgrestore.py hidden away in a webops branch for deploment on non-lp boxes
<cjwatson> Could somebody allocate me a DB patch number for the new job table at the end of bug 745799, if they think it's reasonable in principle?  A summary could just be "Create ProcessAcceptedBugsJob table."
<_mup_> Bug #745799: DistroSeries:+queue Timeout accepting packages (bug structural subscriptions) <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/745799 >
<wallyworld_> rick_h_: hi, i have a yui question. i have widget B that inherits from widget A. A has an ATTR dict with foo, value 1. I want B to override foo to have value 2. I declared an ATTR dict for B with foo, value 2. but doesn't work. any clues?
<rick_h_> wallyworld_: I would think that shold work. Let me put together a test fiddle
<wallyworld_> rick_h_: ok, thanks. i would have though it to be ok too
<rick_h_> wallyworld_: http://jsfiddle.net/r2Hnr/
<rick_h_> wallyworld_: paste/fiddle your code and we can take a look, but it works in that simple case
<wallyworld_> rick_h_: i will need to extract some simple code out, i'll put up a bit of pseudo code
<rick_h_> wallyworld_: ok cool, but yea the concept should work fine
<wallyworld_> rick_h_: ffs. i found it. i was missing a }, {. it was a longish class and syntactically it looked ok but you could only see a bit of the class n the editor
<wallyworld_> the }, { was missing before the ATTR
<wallyworld_> thanks for looking
<wallyworld_> rick_h_: i have an issue for you with the yui version stuff. i tried to run up 3.5 locally, with and without the combo loader feature flag. i changed versions.cfg but stuff just didn't compile properly
<wallyworld_> rick_h_: i wanted to look at Y.panel which was introduced in yui 3.4
<wallyworld_> have you tried to run up 3.5 locally?
<wallyworld_> at one stage we had a feature flag to set the yui version
<rick_h_> wallyworld_: yea, but it wasn't wired in, I've been trying to get that fixed, but let's just say I've learned a LOT about our build/deploy process in the last weeks
<rick_h_> wallyworld_: yea, I've run 3.5 and run all our tests on 3.5
<rick_h_> and at some point hope to get us using it behind a FF, but private projects started up and I'm swamped with wireframe/learning atm
<wallyworld_> rick_h_: cool :-) so tl;dr; is that you do want to get us to the point of a dev being able to run 3.5 locally, but it's a lower priority for now?
<rick_h_> wallyworld_: tl;dr is I've tried landing 3.5 for us on production 3 times with 3 rollbacks. Now a lower priority to take stab #4
<wallyworld_> rick_h_: how about just a dev beiong able to run it locally for playing with?
<rick_h_> wallyworld_: the problem was the buildout script so can't change that until #4 fix gets landed
<rick_h_> hard to do that 'just dev'
<wallyworld_> ok. np. i'll go away now :-)
<rick_h_> no problem, definitely something I want to get done for sure.
<rick_h_> https://code.launchpad.net/~rharding/launchpad/yuiv3/+merge/115592 does it if you want to keep a branch with that patched in
<wallyworld_> ok. cool. thanks
<wallyworld_> i can't wait to use 3.5 stuff :-)
<rick_h_> yea, I'm cranky I didn't get it in there for this new work so I could use it
* abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: frankban* (rvba), abentley | Firefighting: - | Critical bugs: 4.0*10^2
<deryck> rick_h_, ping for standup
<rick_h_> deryck: loading up
<deryck> rick_h_, I think our call will move again.  Timezone conversion fail for the other call.
<rick_h_> deryck: ok, np
<rick_h_> thanks for the heads up
<jcsackett> rick_h_: finally circled back to yesterday's MP. updates pushed.
<rick_h_> jcsackett: rgr, loading up
<rick_h_> jcsackett: did that make sense the whole confusion between arguments, args the method param, and callee.args ?
<jcsackett> rick_h_: for the most part; it's a confusing bit of code honestly. i only understand the portion of it i used, and set the usage notes accordingly. :-p
<rick_h_> jcsackett: yea, that's why I got the headache. I wanted to really figure out wtf was going on
<rick_h_> jcsackett: so thanks for the update r=me
<jcsackett> rick_h_: thanks. :-)
<deryck> rick_h_, just to confirm, since we've moved it around, but our call will be in roughly 45 minutes from now.
<rick_h_> deryck: ok, so normal time
<deryck> yup
<abentley> deryck: quick chat?
<deryck> abentley, sure
<deryck> abentley, standup hangout?
<abentley> deryck: sure.
<abentley> frankban: could you please review https://code.launchpad.net/~abentley/launchpad/better-find-missing-ready-error/+merge/117669 ?
<frankban> sure abentley
<abentley> frankban: thanks.
<frankban> abentley: what do you think about adding an XXX that points to the bug we are trying to diagnose?
<abentley> frankban: sure.
<frankban> cool
<deryck> rick_h_, let's chat now.
<rick_h_> deryck: rgr, made the sound that you joined
<abentley> frankban: pushed.
<frankban> abentley: thanks
<sinzui> jcsackett: do you have time to hangout?
<jcsackett> sinzui: sure.
<rick_h_> sinzui: when you get free can I steal a couple min please?
<deryck> rick_h_, I also added mrevell to our private projects collection.
<deryck> rick_h_, in docs, I mean.  So he can see and edit them.
<rick_h_> deryck: cool, yea I think I set it that all of canonical can edit
<sinzui> rick_h_I am available now
<rick_h_> sinzui: so just quick ? does it ever make sense to have a default information type for an app (say bugs) as public security or private security?
<rick_h_> deryck and I are thinking the defaults (you'd set in an app's edit UI) would always be either public/private and tweak individual items from there
<sinzui> no for 1, we still speculate on the other
<sinzui> rick_h_We do not think there is enough uses to justify those two being defaults
<rick_h_> sinzui: thanks, works in line with what we want to do so <3 that answer
<rick_h_> thanks for the sanity check
<sinzui> rick_h_: We already defined the defaults for bugs...BugSharingPolicy in registry.enums
<rick_h_droid> cool thanks
* frankban changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: abentley | Firefighting: - | Critical bugs: 4.0*10^2
<rick_h_> deryck: ping, think I've got things updated
<deryck> rick_h_, ok, cool.  I'm about to lunch, so will send out email after that.
<rick_h_> deryck: didn't add a ton of new ui stuff based on our conversation aroud project registration, so there's more than appears, but figured it's all be a bunch of fake/time sink
<rick_h_> so there's some text about "Add new block with UI for xx and yy here" since we all know what the UI for person pickers/etc look like
<Bert_2> Hi, because of network destriction I have had to change the download and upload ports in my instance of launchpad from 58080 to 81, 58090 to 82, 58095 to 84 and 85085 to 85, while the links in my templates change perfectly I have the feeling nothing is listening on these ports, does anyone know why ?
<rick_h_> well, I can give up on getteing YUI 3.5 into LP http://yuilibrary.com/
<rick_h_> new goal...3.6 into LP before 3.7 comes out
<deryck> rick_h_, ouch.
<sinzui> rick_h_ why don't you know what the UI for person pickers look like...we wrote that 12 months ago
<rick_h_> sinzui: I was saying we "do" know
<rick_h_> so I didn't cut/paste it into the project signup views
<sinzui> rick_h_okay
<Bert_2> Hi, because of network destriction I have had to change the download and upload ports in my instance of launchpad from 58080 to 81, 58090 to 82, 58095 to 84 and 85085 to 85, while the links in my templates change perfectly I have the feeling nothing is listening on these ports, does anyone know why ?
<sinzui> lifeless: These are the bugs I am watching. In general we are committed to fixing high bugs. We are watching the low for drive-by-fixes or for stakeholder requests to make high: https://bugs.launchpad.net/launchpad/+bugs?field.tag=disclosure
<lifeless> Bert_2: I don't know why you feel that way. Perhaps the first thing to do is to establish what ports things are listening on.
<lifeless> sinzui: how would you like the feedback ?
<lifeless> StevenK: isn't bug 933934 done now ?
<_mup_> Bug #933934: Remove the bug supervisor/security contact subscriptions rules <disclosure> <sharing> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/933934 >
<sinzui> lifeless: comments in the bugs are fine. We might just want to mark several wont fix even though we have not released the replacement behaviour.
<lifeless> I wonder if security contacts can go now.
<lifeless> Entirely.
<lifeless> security grant + structsub.
<lifeless> ?
<sinzui> StevenK: not until we update 1, replace the rules with SS filters for them, then we remove the actual code
<sinzui> lifeless: ^
<lifeless> ok.
<sinzui> lifeless: StevenK: This related to another bug wgrant and I discussed. We want to preserve the communication setup for existing projects. We will document SS+filter as best practice
<RoelV> how does one edit pages after being logged in?
<sinzui> RoelV: which pages?
<RoelV> sinzui: https://dev.launchpad.net/*
<sinzui> you need to be a member of https://launchpad.net/~launchpad-doc
<sinzui> RoelV: have you been using Lp for 3 months or more?
<RoelV> sinzui: yes. userid roel11
<sinzui> RoelV: Ask to join the team, I will approve you a minute later
<sinzui> when you log back into dev.lp.net or help.lp.net, you will have edit rights
<RoelV> sinzui: done thx
<sinzui> RoelV: log back into the wiki
<RoelV> sinzui: great. thx
<rick_h_> anyone recall what buildout uses to determine if it needs to run an install xxx command or not?
<Bert_2> lifeless: basically I tried running on the regular ports, that worked fine but I got yelled at by the network admin I can't run on those ports, so I tried new ones (set them in launchpad-lazr.conf's [librarian] section), but I don't get any images and I get errors on upload, so yeah, it isn't working and I don't understand why
<lifeless> so, check what ports its listening on
<lifeless> and be sure to check the whole stack - apache, squid etc if you have the,
<Bert_2> lifeless: what I'm confused about is that I'm not sure what was listening on 58080 that should now listen to 81
<rick_h_> Bert_2: check the apache config? It has the proxy settings, listening ports, etc
<sinzui> lifeless: would you be available in about an hour to join the purple squad's mumble? The question I will ask is "What are private teams used for and why are they needed?"
<lifeless> sinzui: sure.
<Bert_2> rick_h_: the proxy stuff in /etc/apache2/sites-enabled/local-launchpad are not for port 85080, even though it worked
<Bert_2> lifeless: where can I find the squid configuration ?
<StevenK> lifeless: Filter by information type isn't done yet
<Bert_2> lifeless: my launchpad does not seem to use squid
<lifeless> Bert_2: so, you need to start with the url thats failing, and step by step trace what it should be reaching, and whether that thing is working, etc
<lifeless> Bert_2: this depends entirely on your configuration (not just the lazr conf stuff, your network, front ends, caches, middle ends etc)
<StevenK> CELERY!
 * StevenK stabs buildbot
<Bert_2> lifeless: yeah, the problem is I don't understand what part of launchpad was behind 58080 and is now supposed to be behind 81
<lifeless> neither do we
<lifeless> we don't use either of those ports in production
<Bert_2> but you made the software ? 58080 is the development default :S
<lifeless> for the librarian yes
<lifeless> but the development default has no connection to production use.
<Bert_2> I don't fully understand what you mean by that ?
<lifeless> the development config is made for a single developer on their local machine
<lifeless> not for use in a networked environment
<lifeless> it won't work for a networked environment, the urls aren't sensible
<lifeless> it depends on nasty assumptions that aren't true as soon as you have a real-world situation
<Bert_2> lifeless: I see, so basically image up and download is not possible without reading all the librarian code ?
<lifeless> anyhow, the component that you are asking about is the librarian.
<lifeless> You shouldn't need to read its code; its a straight forward blob store
<lifeless> what are the exact symptoms you encounter ?
<Bert_2> librarian used to work fine when my config had the ports on default, then I changed it to new ports, reran make run and now when I try to load a projectimage it says it can't establish a connection
<Bert_2> so it seems as if librarian is not behind that port
<lifeless> can't establish a connection *to what port*; give us the exact error.
<Bert_2> here's the config: http://pastebin.com/mmGZYwSk
<Bert_2> and when I try to load the image it looks for http://ulaunch.ulyssis.org:81/1/u64.png
<Bert_2> and I get Firefox kan geen verbinding maken met de server op ulaunch.ulyssis.org:81. on that
<Bert_2> which translates to Firefox cannot connect to the server on ulaunch.ulyssis.org:81
<lifeless> ok
<Bert_2> I also tried uploading and it oopsed
<lifeless> so your librarian isn't reachable on that port
<lifeless> have you checked its running and listening on that port?
<lifeless> e.g. using netstat -anp or lsof ?
<Bert_2> lifeless: it seems to be running, but ther's no mentioning of the correct ports in netsta -anp
<Bert_2> and also not of the old ports
<Bert_2> that's very confusing :S
<lifeless> you probably haven't granted it permissions to bind to those ports
<Bert_2> lifeless: how do I do that exactly ?
<lifeless> unix requires privileged access to list to ports <= 1023
<Bert_2> lifeless: yeah, that was why my sysadmin told me I have to stay in that range
<Bert_2> well, netadmin
<Bert_2> lifeless: so, am I supposed to open up port 81 on iptables locally or how do I grant the right ports to librarian ?
<lifeless> Bert_2: There are various docs around for this, its a general problem, not LP specific.
<lifeless> I've got to go now, sorry.
<Bert_2> lifeless: I'll try and find out then
<Bert_2> let's hope some of the others on the team know :p
<Bert_2> thx a ton for your help
<Bert_2> I feel very stupid now XD
<Bert_2> lifeless: I got it, I'm so stupid, I should have known about this, I was focussing on launchpad all the time while obviously the ubuntu user can't catch a <1024 port, thanks a ton for your help and patience
<Bert_2> I'd buy you a beer if you lived in belgium ;)
<james_w> rick_h_, did you fight buildout in to submission? Did you give up for today and leave for the evening? :-)
<rick_h_> james_w: I think I beat it into submissions. I don't know what I did, but it's working now
<james_w> good good
<rick_h_> basically I have a buildout install yui-default and it'd say "upgrading yui-default" but not actually run the script in there.
<rick_h_> it's a plone.recipe.command so running two bash commands in there, but not picked up as if it needed to run them or something. I moved some stuff around, tweaked, and now it's repeatedly working
<rick_h_> where before it would only run if I edited the buildout.cfg in some way
<rick_h_> and I couldn't find what the 'check' was if it needed to execute or just skip the run
#launchpad-dev 2012-08-02
<deryck> I'm starting to worry private projects has too much baggage to be any fun working on.
<deryck> sinzui, are you around?
 * StevenK stabs staging, the update script says that 11801 is the newest, which is a lie.
<wgrant> StevenK: staging doesn't have the new Wgrant-Optimized Update Pathâ¢
<StevenK> :-(
<StevenK> wgrant: I'm just impatient, we have seven hours for staging to update.
 * StevenK stabs the branch scanner too
<wgrant> deryck: I think it shouldn't be too bad once you actually cut through requirements, and everyone works out what needs doing. Disclosure has a similar aura around it, but it proves to be reasonably enjoyable while the path remains clear.
<wgrant> StevenK: Let's see what it's doing..
<deryck> wgrant, I hope you're right.  I'm worried there's too much overlap between our two squads.
<wgrant> We may want to kill the current garbo/karma stuff to speed things along.
<wgrant> Depending on where it is.
<wgrant> StevenK: It started updating 4 minutes ago
<wgrant> StevenK: Should be qa-tagged in maybe an hour.
<wgrant> deryck: There is a lot. But we're almost out of that area.
<wgrant> Hopefully :)
<deryck> wgrant, ah, that's encouraging then.
 * StevenK stabs the branch scanner again
<wgrant> StevenK: What's it doing?
<StevenK> Not scanning ~stevenk/launchpad/bugsubscriptionfilter-itype
<wgrant> StevenK: It's cursed. You probably can't delete it.
<wgrant> StevenK: Rename and repush
<wgrant> StevenK: Did you link a bug manually just after you pushed it?
<StevenK> Yeah
<StevenK> Bleh
<StevenK> So you can't link a bug or create an MP or ....
<wgrant> Not on a Launchpad branch before the initial scan, no
 * StevenK blames BranchRevision
<wgrant> Yes.
<wgrant> StevenK: This scan's not looking good either...
<wgrant> There we go
<wgrant> Success
<StevenK> The UI hasn't updated yet
<wgrant> StevenK: I actually meant you should rename the branch on LP, so you could keep a sensible name.
<wgrant> Has so
<wgrant> You're probably just enslaved.
<StevenK> FInally
<StevenK> s/I/i/
<StevenK> wgrant: And I renamed them around
<wgrant> So I saw
<StevenK> wgrant: Do you want the review, or shall I pick on wallyworld_?
<wgrant> I shall.
<wgrant> While I wait for qastaging to QA for me.
<StevenK> Ah, you have qastaging do your QA for you? That sounds handy, you must tell me how.
<wgrant> StevenK: Did you consider refactoring _set_{statuses,importances,tags,information_types}? There's a lot of ugly common code there.
<wgrant> StevenK: 2012-08-02 02:45:49 INFO    2209-26-3 applied just now in 0.2 seconds
<StevenK> wgrant: I was just pondering how to do that.
<StevenK> wgrant: And thanks
<StevenK> wgrant: I'd love to refactor the three _set_ methods. I just can't think of a way to do it.
<StevenK> Hmmm, maybe
 * StevenK scribbles some notes to fiddle with after lunch
<wgrant> StevenK: They all do a very similar thing. They might want to take a class, an attribute name, the current set of values, and the desired set of values.
<StevenK> wgrant: Yeah, but property() can't do anything with those
<StevenK> So _set_statuses will live on as a one line or so wrapper
<wgrant> Sure, that'd be a helper method.
<wgrant> Exactly.
<StevenK> Since I've got test passing, I'll try my hand at refactoring after lunching
<rick_h_> lifeless: ping, around?
<lifeless> hi
<rick_h_> lifeless: hey, wanted to beg/bribe for a LoC exception for the yui work that's blown up on me a few times.
<rick_h_> first, fixing the buildout recipe upstream seemed dead since there's no upstream I can find and it's not maintained since 2010: http://pypi.python.org/pypi/plone.recipe.command/
<rick_h_> lifeless: so in stand up it came up about just doing the unzip work in a python script vs using the unzip dep in the buildout command
<rick_h_> obviously, adding a python script to do the work is aLoC increase, but the goal of the work is to allow the YUI version feature flag to work and allow easily testing YUI/maintaining the JS state in the build directory
<rick_h_> so wanted to see if I could convince you better for maintance and hopefully doesn't hang production servers on deploy this time :/
<rick_h_> not done yet, but start of the idea: https://code.launchpad.net/~rharding/launchpad/yuiv4/+merge/117800 doing in spare time around the private project requirements stuff
<lifeless> whats the copytree for ?
<rick_h_> lifeless: the .zip has test/doc dirs we don't want. We only care for the build dir. so we extract to a build/js/tmp and copytree the build dir to build/js/yui-xxx
<lifeless> can't you just pass in the dir you want ?
<lifeless> hmmm, doesn't let you transform it
<lifeless> sadface.
<rick_h_> zipfile will let me extract based on name, but I need to have all the names I want not a single dir adjusted
<lifeless> ok
<lifeless> rick_h_: so, plone.recipe.command is still buggy here, I'm worried about us using it. It is a booby trap.
<rick_h_> lifeless: true, I wanted to ditch it but don't see how to get the version into the here from versions.cfg and such directly
<lifeless> rick_h_: I think the loc overhead is tolerable, but that about 80% of the overhead is because you're calling a process not a function.
<lifeless> 'version into the here' ?
<rick_h_> lifeless: so yui-default gets the version from versions.cfg, passed into the call to this script via the buildout section
<rick_h_> my first thought was to use the buildout generation to hard code the yui version into the script when the template is extracted
<rick_h_> but that only gives me one version, not multiple for the feature flag setting
<lifeless> versions.cfg talks about python packages
<lifeless> yui isn't a python package.
<lifeless> How is it relevant ?
<rick_h_> yes, but part of the reason my first idea was turned down was that it didn't respect versions.cfg. There's an entry in versions.cfg for YUI that is used to calc the 'default' yui everyone gets
<lifeless> why?
<rick_h_> because it's a nice single place to specify the correct version of a dependency? I'm only guessing at the original reasong
<rick_h_> reasoning that is
<rick_h_> if it wasn't for that I'd turn the whole thing into a make target and skip buildout entirely, but that versions.cfg traps me into the use
<lifeless> don't use versions.cfg
<lifeless> easy done.
<rick_h_> ok, nvm then. I should be able to continue to use unzip from the makefile then, ditch the plone.recipe.command all together, and rework this
<lifeless> seriously, versions.cfg is for python package versions, its not rich enough to describe '4 versions of a package'
<lifeless> because python doesn't support concurrent loading of one API
<rick_h_> right, that's why the buildout is set to run yui-default and yui-3.5
<lifeless> so to wedge this into versions.cfg, you'd have to use N different packages
<rick_h_> lifeless: ok thanks, works for me.
<lifeless> and do string matching to decide which ones were YUI
<lifeless> -> thats more than a little horrid
<rick_h_> well, it appears to populate a var yui_version = ${versions:yui}
<rick_h_> so it didn't seem quite that dirty, but understand
<lifeless> I see 'yui = 3.3.0' in versions.cfg
<lifeless> what am I missing ?
<rick_h_> lifeless: right, the yui-default block in buildout.cfg is getting that via ${versions:yui}
<lifeless> sure, but this does demonstrates the friction.
<rick_h_> I don't think you're missing anything, just that we're only using it to describe the 'default' version of the package and using additional buildout.cfg blocks to load the feature flag available versions of YUI for side by side install
<lifeless> You're encoding versions in buildout.cfg
<lifeless> not in versions.cfg
<rick_h_> lifeless: correct, lots of friction
<rick_h_> lifeless: right, ok. I'll rework it in my spare time and make an effort to ditch the plone.recipe.command
<rick_h_> thanks for the time/chat.
<lifeless> anytime
<lifeless> long term the mythical handling-js system will want this flexability
<rick_h_> yes, with 3.6.0 released today it's killing me I've not gotten it sorted yet.
<StevenK> wgrant:  2 files changed, 40 insertions(+), 75 deletions(-)
<wgrant> StevenK: Excellent.
<StevenK> wgrant: http://pastebin.ubuntu.com/1124649/
<wgrant> StevenK: _get_collection, _set_collection, mabe?
<wgrant> maybe
<StevenK> wgrant: _set_collection makes the line 78 lines
<StevenK> Er, chars
<wgrant> That's fine.
<StevenK> I thought 77 was the limit?
<wgrant> 80
<wgrant> now
<wgrant> Or was it 79
<wgrant> One of those two
<wgrant> Certainly not 77
<wgrant> /usr/share/pyshared/pocketlint/formatcheck.py:DEFAULT_MAX_LENGTH = 80
<wgrant> sinzui has spoken.
<StevenK> wgrant: Right. Do you want to see the new diff?
<wgrant> StevenK: That would be lovely.
<StevenK> wgrant: http://pastebin.ubuntu.com/1124660/
<wgrant> Now it's a little easier to see the differences :)
<StevenK> wgrant: Shall I push that up?
<wgrant> StevenK: Might as well.
<wgrant> Less good is better code.
<wgrant> Er
<wgrant> Less code is better code.
<StevenK> Yes, I'm not sure I want to refactor _get_tags() and friends
<StevenK> I'm tempted to kill the pointless docstrings
<StevenK> wgrant: Opinion?
<wgrant> Maybe
<StevenK> wgrant: Killing the docstrings turns it into +39/-100
 * StevenK stabs Firefox
<StevenK> lifeless: So I'd like to organise an FDT tonight with two patches, 2209-26-{2,3}.
<StevenK> lifeless: 26-2 updates two tables, BugSubscriptionFilter{Status,Importance} and 26-3 creates a new table.
<wgrant> More specifically, 26-2 changes from a serial to a natural primary key
<StevenK> lifeless: Timing on staging puts both patches applying in 0.5 seconds.
 * StevenK sorts out the db-stable deployment report
<StevenK> wgrant: Do we want a NDT?
<StevenK> I'd like to make sure I've not broken get_bug_tags() with r15733, but I'm not sure which pages make use of it.
<wgrant> StevenK: Let's see.
<wgrant> StevenK: Huh
<StevenK> wgrant: Hm?
<wgrant> StevenK: I may have missed something. What do you see as the callsites for it?
<wgrant> I can only see one, but I'm presumably missing something.
<StevenK> wgrant: IDistroseries.getUsedBugTags(), IDistribution.getUsedBugTags(), IProduct.getUsedBugTags() at least
<StevenK> IProductSeries and IProjectGroup too I think
<wgrant> Right. And what uses those?
<wgrant> Yeah
<wgrant> That whole set of methods can just be deleted
<wgrant> Only one callsite
<wgrant> (BugEditView uses it to decide whether a tag is new. But the JS doesn't do it, so the classical form doesn't need to either)
<StevenK> So I fixed them for nothing? :-)
<wgrant> Well
<wgrant> It led us to them
<wgrant> So now we can destroy them.
<StevenK> Which is the callsite I need to preserve?
<wgrant> There is only one callsite that I can see, BugEditView.validate
<wgrant> And you don't need to preserve it
<StevenK> Uh?
<StevenK> lib/lp/registry/model/distributionsourcepackage.py:        return self.distribution.getUsedBugTags()
<StevenK> lib/lp/registry/model/sourcepackage.py:        return self.distroseries.getUsedBugTags()
<wgrant> I'm pretty sure those are just delegations
<wgrant> But I haven't checked.
<StevenK> lib/lp/bugs/browser/bug.py:            bugtarget.getUsedBugTags() + bugtarget.official_bug_tags)
<wgrant>     def getUsedBugTags(self):
<wgrant>         """See `IBugTarget`."""
<wgrant>         return self.distribution.getUsedBugTags()
<wgrant> StevenK: Right, that one is the only real callsite
<wgrant> And it can die
<StevenK> wgrant: It can?
<wgrant> Look at what it does.
<wgrant> (ignoring the XSS hole)
<StevenK> So the entire validate method can just die horribly?
<wgrant> Right
<wgrant> You might have noticed the new tag validation on the AJAX widget.
<wgrant> It's very fancy.
<wgrant> So fancy that it's invisible and 0 bytes in size.
<StevenK> Hahaha
<wgrant> Since everyone has used the AJAX widget since 2009 and not lamented this warning's absence, I think it can be abolished.
<wgrant> as a bonus, you get rid of shitty code
<wgrant> And an XSS vulnerability or two
<wgrant> There's also a not insignificant volume of tests.
<wgrant> So whoever does this gets a nice LOC credit.
<lifeless> StevenK: got a bazaar.l.n link to those two patches ?
<StevenK> lifeless: Sure, give me a tick.
<wgrant> StevenK: :( you stole my patch series
<wgrant> http://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-devel/view/head:/database/schema/patch-2209-26-2.sql http://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-devel/view/head:/database/schema/patch-2209-26-3.sql
<wgrant> 2209-26-* was a new series so the BVP stuff was grouped :(
<StevenK> I didn't steal them, they were freee
<wgrant> lifeless: ^^ the two patches I linked above
<StevenK> wgrant: 13 files changed, 5 insertions(+), 150 deletions(-)
<lifeless> ok
<wgrant> StevenK: You've grepped around for tests?
<lifeless> StevenK: I'm curious why a new table not an array
<wgrant> And any other orphaned methods?
<wgrant> lifeless: I suggested a new table, to maintain consistency and share code.
<wgrant> lifeless: But indeed, all four filters deserve to be arrays
<wgrant> And should probably be migrated that way eventually.
<StevenK> wgrant: Yeah
<StevenK> steven@undermined:~/launchpad/lp-branches/destroy-getusedbugtags% bzr grep getUsedBugTags | grep -v getUsedBugTagsWithOpenCounts | wc -l
<StevenK> 0
<lifeless> StevenK: you have+1 on double-dipping the FDT.
<StevenK> lifeless: Thanks
<StevenK> wgrant: So I guess FDT for r11803
<wgrant> StevenK: You also removed confirm_tag_action?
<StevenK> Nope
<wgrant> And indeed, 11803 is correct
<wgrant> But you should remove confirm_tag_action
<wgrant>     _confirm_new_tags = False
<wgrant> can die as well
<wgrant> And render()
<wgrant> And __init__
<wgrant> And part of the template.
<wgrant> s/part/all/
<wgrant> That view appears to be nearing its end...
<StevenK> Which template?
<wgrant> The view's template.
<wgrant> It's a form
<wgrant> Except for the notifications
<wgrant> Except we've removed the notifications
<wgrant> So it's a form.
<wgrant> Oh
<wgrant> hm
<StevenK> So I drop the template from the ZCML?
<wgrant> The action can go away as well
<wgrant> Since that just makes it a LaunchpadEditFormView
<wgrant> So the view ends up with like one method
<wgrant> Right
<wgrant> Um
<wgrant> Maybe
<wgrant> May need to use generic-edit-form.pt or whatever it is
<wgrant>         template="../../app/templates/generic-edit.pt"/>
<StevenK> wgrant: 15 files changed, 9 insertions(+), 230 deletions(-)
<wgrant> StevenK: Make sure to set next_url properly
<wgrant> I'd rename cancel_url to next_url, then set cancel_url = next_url
<StevenK> Already done
<wgrant> So the view's like 10 lines and uses the generic template now, right?
<StevenK> wgrant: Hmmm, it is 12
<StevenK> BugMarkAsDuplicateView had a little surgery too
<wgrant> StevenK: What'd you do to it?
<StevenK> wgrant: It had next_url specially due to BugEditView, now that it doesn't require the specialness, it can be pushed up to the base class
<wgrant> StevenK: Ah, right
<wgrant> StevenK: However, I've just noticed what may be a fatal flaw in this simplification
<wgrant> StevenK: They're views on BugTask
<wgrant> But they operate on the Bug
<wgrant> That's why they're not classical LEFVs, I suppose.
<wgrant> So you may need to preserve the custom actions.
<StevenK> Hmmmm, not sure.
<StevenK> wgrant: 5 failed tests
<wgrant> StevenK: :(
<StevenK> A few of them look caused by my overzealous deletion
<wgrant> Nah
<wgrant> No such thing as overzealous deletion
<StevenK> Haha
<wgrant> It just means you haven't deleted enough tests
<wgrant> => underzealous deletion
<adeuring> good morning
<dpm> hi, could someone help me? I'm trying to build the language packs for 12.04.1, from a script running on a server which calls LP (the getPackageUploads API call) to fetch some translations files. However this seems to fail with a timeout, I've tried a couple of times with the same end result. Here is the log from the script, and the LP response. Perhaps the 'x-lazr-oopsid: OOPS-80db179b422b1ae941c44605effcdaf1' could be helpful in determining what's hap
<dpm> pening?
<dpm> Any help will be greatly appreciated, as this is currently blocking the 12.04.1 language pack build
<czajkowski> adeuring: wgrant StevenK anyone able to help dpm out
<dpm> thanks czajkowski
<dpm> sorry, forgot to paste the log, here it is: http://pastebin.ubuntu.com/1124842/
<wgrant> This is probably fallout from cjwatson's work, but let's see...
<dpm> I'm running the script again, to see if it runs through or if I get another timeout
<dpm> wgrant, have you had a chance to look at the timeout issue I was mentioning (not sure if you meant you were looking at it, just checking)? ^
<wgrant> dpm: Oh, I think we got netsplit
<wgrant> 19:10:23 < wgrant> Yeah
<wgrant> 19:10:35 < wgrant> The new PackageUpload security adapter is issuing several queries for every item.
<wgrant> 19:10:36 < wgrant> Um
<wgrant> 19:11:05 < wgrant> dpm: Where's this script?
<dpm> wgrant, ah, yeah, hadn't seen the ping. So the script is at http://bazaar.launchpad.net/~ubuntu-langpack/langpack-o-matic/main/view/head:/import and the relevant bit that calls the function is at http://bazaar.launchpad.net/~ubuntu-langpack/langpack-o-matic/main/view/head:/lib/static_translations.py#L58
<StevenK> webops: Who is my willing vict^W^Wfriendly webop for the FDT in 25 minutes?
<StevenK> Sigh
<mthaddon> StevenK: I am
<wgrant> dpm: Ugh, that's pretty awful. Grabbing like 120000 uploads from LP every time.
<wgrant> Let's see.
<dpm> ouch
<wgrant> Ah, it's actually several times more than that.
<dpm> wgrant, anything that can be done either in LP or in the script to improve on that and avoid the timeout?
<wgrant> dpm: Sorry, will get back to you soon, just had a bit of a crisis.
<dpm> wgrant, no worries, thanks for your help
<wgrant> jam: re. dpm's issue, I'd like to bump the timeout for DistroSeries:EntryResource:getPackageUploads from 5s back up to 9s, as it was a couple of months back.
<wgrant> It'll hopefully be enough.
<wgrant> Just.
<dpm> wgrant, I assume that even if the timeout is reverted in LP, it will still take a while for it to land. Is there anything I can do on my side (i.e. on the script), so that I can start building the language packs in the meantime?
<wgrant> dpm: We can apply the timeout change instaneously.
<wgrant> Once I find a TL to approve it.
<dpm> wgrant, ah, awesome, that'd work best for me then if it's possible
<StevenK> Oh look, there's one.
<jam> wgrant: it seems a pretty major band-aid, since once he needs another 10% more data it will hit the timeout again, but if it unblocks him for now, then it seems ok in the short term.
<jam> Is there a way to comment the change so we know who needs the time bump?
<wgrant> jam: Well, it's batched
<wgrant> jam: Batches of 40 work atm
<wgrant> Batches of 75 do not
<wgrant> An O(batch size) query count is the problem.
<wgrant> And it's easier to set a feature flag on LP temporarily than hack launchpadlib or the script
<jam> wgrant: it is batched inside launchpadlib?
<wgrant> jam: Right, iterating over a collection retrieves it in chunks
<jam> wgrant: so do we have a decent way to document the decision, so we can revisit it later?
<jam> or is there at least a bug about DistroSeries:EntryResource:getPackageUploads being O(batch_size) ?
<jam> if so, it seems fine to bump the timeout
<wgrant> jam: I'm going to talk to cjwatson about it, but there indeed needs to be a bug.
<wgrant> jam: We can mention in the feature flag changelog that it's because of this script
 * wgrant is filing
<wgrant> Bug #1032176
<_mup_> Bug #1032176: DistroSeries.getPackageUploads timing out due to authorization checking  <api> <regression> <soyuz-core> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1032176 >
<wgrant> jam: So, I propose to add the feature rule 'hard_timeout pageid:DistroSeries:EntryResource:getPackageUploads 120 9000'
<jam> wgrant: what is the 120?
<wgrant> jam: The priority
<wgrant> We currently have up to 119
<wgrant> https://launchpad.net/+feature-rules
<jam> wgrant: seems fine to me
<wgrant> jam: Thanks.
<wgrant> dpm: Care to try the script again?
<wgrant> The timeout is up and the method call works for me now
<sinzui> jcsackett: we are going to hangout with Orange and 16:00. I want their help to solve sharing parts of projects
<dpm> wgrant, sure, let me start the run, thanks!
<dpm> running now
<dpm> last time it stopped after ~10 minutes so I guess if it runs longer than that that'll be some indication that it works
<jcsackett> sinzui: i assume from the use of 24 hour time you mean UTC?
<sinzui> jcsackett: yes, sorry for the ambiguity
<jcsackett> sinzui: no worries. :-)
<barry> hi folks.  we have a rather urgent problem with a spammer: https://answers.launchpad.net/launchpad/+question/204862
<barry> can anyone help us?
<czajkowski> barry: sure
<czajkowski> barry: done
<barry> czajkowski: thanks!  i know we'll continue to play whack-a-mole with this person.  this isn't the first time and i'm sure it won't be the last.
<barry> i don't know why he hates us ;)
<czajkowski> meh always one
<czajkowski> just file a new answer each time and we can track it
<barry> czajkowski: thank you
<czajkowski> np
* abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
<abentley> adeuring: There's no OCR.  Could you please review https://code.launchpad.net/~abentley/launchpad/better-find-missing-ready-error/+merge/117926 ?
<adeuring> abentley: sure
<adeuring> abentley: r=me, minor nitpick
<abentley> adeuring: Thanks.
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: jcsackett | Firefighting: - | Critical bugs: 4.0*10^2
<jcsackett> abentley: apologies for not having set the topic earlier.
<abentley> jcsackett: I'm a slacker too.  I was OCR yesterday and today I was still listed in the topic.
<jcsackett> the topic is the easiest part to forget. esp if you get in the habit of just checking the activereviews page all the time.
<lifeless> o/
<rick_h_> deryck: ping
<deryck> rick_h_, hi
<lifeless> deryck: hi, so I can do a call whenever
<rick_h_> deryck: got a sec to chat before I EOD?
<deryck> rick_h_, on call, sorry.
<rick_h_> deryck: np
<deryck> lifeless, sorry, crazy afternoon.  Let's chat next week.  Too much churn right now with what's going on with purple too.
<lifeless> ok
<jcsackett> irssi
<lifeless> indeed
 * jcsackett just now realizes that the "dead" irssi window he was restarting wasn't dead.
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
#launchpad-dev 2012-08-03
<StevenK>          BugSubscriptionFilterInformationType.information_type ==
<StevenK>              bug.information_type,
<StevenK> -        BugSubscriptionFilterStatus.status == None))
<StevenK> +        BugSubscriptionFilterInformationType.information_type == None))
<StevenK> wgrant: ^ Spot the error.
 * StevenK puts a brown paper bag over his head
<wgrant> Heh
<wgrant> StevenK: Does that fix everything?
<StevenK> wgrant: Yup
<wgrant> StevenK: Excellent.
<wgrant> StevenK: Is it in devel yet?
<StevenK> wgrant: No, I'll be lp-landing it in a few moments.
<wgrant> Great.
<StevenK> wgrant: r15739
<wgrant> Indeed.
<StevenK> Hmm,     [action.label for action in bug_edit.view.actions]
<StevenK> Is [] rather than ['Change']
<wgrant> StevenK: There are some issues with inheritance of actions
<wgrant> I don't know details, but I've seen workarounds.
<StevenK> I'm waiting for make run so I can poke
<StevenK> Hahaha, there is no button
<wgrant> bin/run -r librarian -i development
<StevenK> wgrant, wallyworld_: https://code.launchpad.net/~stevenk/launchpad/destroy-getusedbugtags/+merge/118024
<wgrant> StevenK: Already there
<wgrant> +15/-287 is what I like to see
<wgrant> StevenK: Why'd you remove the wasRedirected check on line 164 of the diff?
<wgrant> It may be legit.
<StevenK> The test failed with it in
<wgrant> Right, but it probably shouldn't have.
<StevenK> Why? That was the screwy redirect back to ourselves if the tags need confirming?
<wgrant> That would be a non-redirect
<wgrant> The redirect is back to BugTask:+index
<wgrant> The form seems to work fine, but that test should probably pass
<StevenK>     bug_edit.wasRedirected()
<StevenK>     - True
<StevenK>     + False
<wgrant> StevenK: Oh
<wgrant> Look at that block of the test
<wgrant> It needs to die
<wgrant> And you want to rewrite the prose in that whole section
<wgrant> 658-672 of that file can probably just be deleted
<wgrant> And the prose before that mentions adding a new tag
<wgrant> Which isn't special any more
<StevenK> http://pastebin.ubuntu.com/1126322/
<wgrant> StevenK: You have a blank line where there should not be a blank line, but otherwise good
<StevenK> wgrant: Any other comments?
<lifeless> wallyworld_: / wgrant: bug https://bugs.launchpad.net/launchpad/+bug/971144
<_mup_> Bug #971144: Remove direct sharee on sharing audit page should show any remaining team access <disclosure> <sharing> <ui> <Launchpad itself:Won't Fix by wallyworld> < https://launchpad.net/bugs/971144 >
<lifeless> it claims there *is* a sharing audit page now, and that it expands teams.
<wallyworld_> there was but it was removed ages ago
<lifeless> On the call you guys said that there was no such thing.
<lifeless> ah, wontfix. I see (I was blind. bah).
<lifeless> thanks
<wallyworld_> np
<wgrant> Well
<wgrant> There was a page
<wgrant> It was never finished
<wgrant> And didn't live for long
<wgrant> StevenK: r=me, thanks
<StevenK> wgrant: Finally tossed at ec2, thanks
<wallyworld_> lifeless: in canonical_url, why do we only allow specifying the rootsite when request is None? In the case where a request comes in from code.lp.net and you want to get a bug url, it is not possible since the result starts with "code.lp..."
<lifeless> I don't know
<lifeless> I haven't looked at that in a year o rso
<lifeless> there should be tests with attached rationale
<StevenK> canonical_url() is useful, but horrible
<wallyworld_> ok, will poke around
<wgrant> wallyworld_: Why're you passing in a request?
<wgrant> Oh, to get a webservice base?
<wallyworld_> yes
<wallyworld_> i want to get self_link for a bug
<wgrant> wallyworld_: That's not really what you want to do.
<wgrant> wallyworld_: since cross-domain AJAX requests == boom
<wgrant> The problem is that you *are* getting a bugs URL
<wgrant> You need to not get one
<wallyworld_> but i want one
<wgrant> You might want one for the web_link
<wgrant> But I don't see why you'd want one for self_link
<wallyworld_> i need it to pass back to the client
<wallyworld_> so the client can invoke an api on the bug
<wgrant> The JS client can't use a self_link that's on another domain
<wallyworld_> it does now i think
<wgrant> Nope
<wgrant> Same-origin ensures that it can't
<wallyworld_> ah, it passes it in as a param
<wallyworld_> so i need the self_link to pass as a param so that lazr restful knows how to unmarhsall it as a bug
<wallyworld_> the js client now makes an extrs xhr call to get this info
<wgrant> You need to either use the path (eg. '/bugs/1234', or the path prefixed with the webservice root that the method is on
<_mup_> Bug #1234: Gina is an unmaintainable mess of command line options, environment variables and shell scripts <lp-foundations> <Launchpad itself:Fix Released by debonzi> < https://launchpad.net/bugs/1234 >
<wgrant> You can't normally pass a bugs.launchpad.net URL into a method call on code.launchpad.net
<wallyworld_> it does now in some form, i'll check exactly what is passed
<wallyworld_> huh, so atm we issue a GET to http://code.lp.net/api/devel/bugs/1235
<_mup_> Bug #1235: When batch_start is >= the number of bugs shown, an error is raised <lp-foundations> <oops> <Launchpad itself:Fix Released by spiv> < https://launchpad.net/bugs/1235 >
<wallyworld_> and the result has all the api links starting with code.
<wgrant> Right.
<wallyworld_> which seems wrong but anyway
<wgrant> Domain doesn't matter in terms of API interpretation
<wgrant> It only exists on the webapp subdomains to work around same-origin
<wgrant> s/It/The API/
<wallyworld_> ok
<wgrant> Yes, it is hideous, that's why I one-domainified LP in November :)
<wallyworld_> so what happened to stop deploying it?
<wgrant> Probably a combination of politics, leave, and teams being reassigned elsewhere.
<wallyworld_> that's a shame
<lifeless> wgrant: was it ready to action ?
<wgrant> lifeless: There's one minor remaining technical issue: Product:+code-index is subtly different from Product:+branches in ways that confuse users and prevent trivially merging of them
<wgrant> Everything else was ready to land.
<lifeless> stub: hi, skype ?
<stub> lifeless: sure
<stub> its firing up, finding my mike
<lifeless>  I want to be like mike
<adeuring> good morning
<lifeless> wgrant: https://launchpad.net/anonymiseip
<wgrant> lifeless: brb, rainbow table
<lifeless> wgrant: acknowledged in the README
<lifeless> wgrant: you don't need one, its only 32 bits.
<wgrant> Ah, in the middle of a 12 line paragraph
<wgrant> lifeless: Meh, still more efficient if I want to steal all your users' IPs
<lifeless> well yeah.
<lifeless> So, if when we get ipv6, will need a deeper algorithm. But at that point it might actually be worth it.
<lifeless> 32 bit, anything deterministic is brute forcable if you can get a chosen plaintext through it.
<lifeless> E.g. hit a bad url from your own ip
<wgrant> Hmm? A salt would work fine unless you want to bruteforce via the webservice.
<lifeless> you toss your chosen plaintext in and grab back the anonymised log entry matching the chosen URL
<lifeless> that gives you the salted output for your ip
<wgrant> How does that help you?
<wgrant> It's salted.
<lifeless> bruteforce that - only 16 million to check
<wgrant> That's a pretty shitty salt
<lifeless> huh, its the output modulus that matters
<lifeless> doesn't matter how big the salt
<lifeless> if its going through a fairly flat function like w.g. sha1, then the bit reduction means you're not going to be far off of 2^32 before you find a hit
<wgrant> I'm still not seeing the attack that you describe.
<wgrant> But food calls.
* adeuring changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: adeuring | Firefighting: - | Critical bugs: 4.0*10^2
<cjwatson> lifeless: I'd appreciate a more detailed pointer regarding https://code.launchpad.net/~cjwatson/launchpad/archive-getallpermissions/+merge/117606, if you have time - in particular, I'm not sure where/if invisible teams are magically filtered out of the returned collection right now
<cjwatson> It seems most sensible to push it down into the query; but it is odd that Person._teamPrivacyQuery is a private method (well, insofar as Python has those), since that looks like the ideal thing to call
<cjwatson> Could somebody allocate me a DB patch number for the new job table at the end of bug 745799, if they think it's reasonable in principle?  A summary could just be "Create ProcessAcceptedBugsJob table."
<_mup_> Bug #745799: DistroSeries:+queue Timeout accepting packages (bug structural subscriptions) <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/745799 >
* bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: adeuring, bac | Firefighting: - | Critical bugs: 4.0*10^2
<bac> hi frankban
<czajkowski> jml: did you get my reply to the list
<czajkowski> mails from me seem to be taking forever and a day to go to lists :/
<jml> czajkowski: yep
<jml> czajkowski: wait... you replied directly to me
<jml> czajkowski: as well as to the list
<czajkowski> I replied to the list
<czajkowski> not to you I always reply to lists
<jml> oh quite right.
<jml> then yes, I got it.
 * jml is so used to Reply To All 
<czajkowski> any reason you were wondering :)
<jml> czajkowski: yes, actually
<jml> czajkowski: james_w overheard on #launchpad-ops that there's downtime scheduled for Aug 17. I was wondering whether that was a further rescheduling of the cocoplum downtime that happened yesterday
<czajkowski> different..
<jml> czajkowski: yeah, so I gather.
<jml> czajkowski: other, less vigilant stakeholders would probably appreciate an email to the list about it.
<wgrant> I imagine everyone will be getting details about it soon
<czajkowski> jml: I'm sure we will when they are ready
<wgrant> We don't know specifics yet
<jml> wgrant, czajkowski: glad to hear it. thanks.
<rick_h_> jcsackett: morning, off the stand up so ping when you're ready to chat.
<jcsackett> rick_h_: ping.
<rick_h_> jcsackett: pong
<jcsackett> rick_h_: g+?
<rick_h_> jcsackett: yea, starting one up now
<jcsackett> bac: could you look at https://code.launchpad.net/~jcsackett/launchpad/info-type-from-main-host/+merge/118128
<bac> jcsackett: done
<jcsackett> thanks, bac.
<bac> jcsackett: hope it wasn't too verbose
<jcsackett> bac: just right.
<jcsackett> rick_h_: ping me when you're available to chat. i'm free now.
<rick_h_> jcsackett: awesome, I'll set it up
* bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
#launchpad-dev 2012-08-04
<lifeless> cjwatson: hi
<lifeless> cjwatson: lazr.restful walks the declared attributes of its results
<lifeless> cjwatson: catching permission errors and eliding those things, whatever they are.
<cjwatson> OK
<lifeless> cjwatson: so for a collection, it materializes the batch, then iterates
<lifeless> if you have objects that have permission checks which require db lookups, you end up calling whatever logic is needed for each object -> very slow.
<cjwatson> Got it, I think.  EntryResource.redacted_fields?
<lifeless> Person._teamPrivacyQuery is likely what you want to use, however I'm stale on the detail; suggest you consult with purple (e.g. wgrant) about the current state of the art
<wgrant> lifeless, cjwatson: Well, this is an... interesting... case
<wgrant> Times you don't want to filter stuff out:
<wgrant>  - auditing permissions
<lifeless> cjwatson: So, I haven't read the plumbing in lazr.restful, but that sounds likely.
<lifeless> wgrant: interacting w/public object -> discloses team.
<wgrant> lifeless: Sure
<wgrant> lifeless: this is an interaction with a public object
<lifeless> wgrant: so if we add the rule, add it to the relevant core query, then join back on that.
<wgrant> So it discloses the team
<wgrant> So filtering based on privacy is wrong
<wgrant> lifeless: no, that is impractical
<wgrant> We can't just add all 90 FKs to the privacy check
<wgrant> However
<wgrant> I suspect that private team linkage checks might forbid ArchivePermissions
<cjwatson> Is this a case where you should be forbidden from creating such APs in the first place?
<wgrant> Right, I suspect it may already be
<wgrant> Although the vocab is ValidPersonOrTeam, so maybe not
<wgrant> lifeless: The private team security adapter is already very slow. We need to make it faster, not slower.
<cjwatson> I rather suspect there are none right now and we could (a) check that quickly (b) forbid it henceforth.
<wgrant> We eventually shouldn't forbid it, but it's the right thing to do now, until we have private team privacy sorted out
<lifeless> so derived distros will want private teams
<cjwatson> I'm not entirely convinced that a public archive would ever want to have upload permissions associated with it for a private team.
<lifeless> do AP's apply to ppas ?
<wgrant> lifeless: They do.
<lifeless> and we have private teams with private ppas
<wgrant> Yes.
<wgrant> But they don't necessarily have APs
<wgrant> For PPAs, the archive owner has implicit access
<lifeless> so I believe this invalidates any assertion about public being the only valid choice in the system
<cjwatson> I didn't assert that
<cjwatson> Would the owner of a private PPA want to grant upload permissions to a team whose membership they can't see?
<cjwatson> Surely not.
<wgrant> cjwatson: They can see it
<lifeless> cjwatson: no, but thats != not using private teams
<wgrant> cjwatson: Because they're in the team
<cjwatson> Indeed
<lifeless> or they own it
<cjwatson> Hence showing that team => not a problem
<lifeless> and the implicit visibility rule says that they should be able to see it afterwards anyway.
<cjwatson> I assert that if you can see the archive, you should be able to see the teams with permissions to upload to it
<lifeless> so its fine, I think, to say that this API can force showing everything.
<wgrant> cjwatson: The rule is that you can see the name, displayname and owner
<wgrant> cjwatson: You can't see the membership
<lifeless> but we need the reflexive side to work as well.
<cjwatson> And that if that isn't true, we've made a mistake in allowing those permissions to be granted in the first place
<lifeless> which is to say that if you see a team in the API you need to be able to see the team directly as well (limitedview)
<cjwatson> (Note this API is only usable by people who can see the archive)
<lifeless> sure
<lifeless> I think wgrant and I are riffing about how to make sure the reflexive side works, more than anything.
<wgrant> And we have strongly disagreed on the approach in the past :)
<lifeless> that, and that the embedded team object in the API result needs to be appropriately neutered.
<wgrant> lifeless: The team shouldn't be embedded, I don't think
<lifeless> wgrant: think of it as a challenge to your ingenuity
<wgrant> It'll include a link
<wgrant> And should only check LimitedView for that
<lifeless> wgrant: I'd like to be confident of that
<wgrant> lifeless: Well
<lifeless> wgrant: a test, even if thrown away afterwards, would do that.
<wgrant> The way you become confident is to only grant LimitedView
<wgrant> Not View
<lifeless> wgrant: sure, +1
<cjwatson> So, aside from the angle of avoiding unwanted disclosure or not, lifeless pointed out that not doing some kind of privacy check up front appears to be a performance issue
<wgrant> Well
<lifeless> history tells us :>
<wgrant> As long as you preload the people themselves, it won't be a performance issue unless there are private teams.
<cjwatson> If we're not filtering, then presumably we at least need to preload
<cjwatson> Right
<lifeless> there is a 'Person' eager loader which does the right thing for teams too
<wgrant> (Since the privacy check on any person other than a private team is a trivial inline attribute access)
<lifeless> so using that + injecting LimitedView onto everything should be sufficient
<cjwatson> i.e. precache_permission_for_objects?
<wgrant> Right
<cjwatson> wgrant: Does my proposal in bug 745799 look awful?
<_mup_> Bug #745799: DistroSeries:+queue Timeout accepting packages (bug structural subscriptions) <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/745799 >
<cjwatson> It's kind of a hack around structsubs being insane, but ...
<wgrant> cjwatson: "awful"? Yes. "necessary anyway"? Probably. I need to look at related structsub performance issues with StevenK on Monday, but I suspect the fix will be sufficiently uninvasive that it doesn't fix your issue.
<cjwatson> The current implementation is sufficiently recursive that I couldn't see a way to improve it without ripping much of it to shreds
<wgrant> Yeah
<wgrant> It's reasonably likely that the refactoring that we require will make it substantially less bad.
<wgrant> But it probably won't be reliably good enough for your uses.
<cjwatson> Seeing as the list of bugs to close can be arbitrarily long, indeed
<wgrant> Right
<wgrant> We're only really interested in optimising for the single bug case.
<wgrant> And that's all that's feasible to do at this stage without massive refactoring.
<cjwatson> If you'd feel like allocating me a DB patch number, I can start with moving the review process for this into merge proposals and see what people think there
<wgrant> cjwatson: Why's distroseries nullable/
<wgrant> You could interpret that to mean upload_distroseries
<wgrant> But that seems pointless.
<cjwatson> cf. close_bugs_for_sourcepackagerelease
<cjwatson> It's upload_distroseries in my current imp
<cjwatson> Because otherwise I had to pass either a DSP or an SP, or whatever the objects are
<wgrant> cjwatson: Sure, but I don't see much benefit in having the job use SPR.upload_distroseries, rather than forcing the job creators to always pass a series.
<wgrant> SPR.upload_{distroseries,archive} are abominations. We should restrict their callsites as much as possible.
<cjwatson> Ah, well that's possible yes
<cjwatson> It's just close_bugs_for_queue_item, and that could use queue_item.distroseries
<cjwatson> No idea why it doesn't
<wgrant> Right
<wgrant> It's only the initial non-copy PackageUpload stuff which should be using upload_* at all
<wgrant> Everything else should use it from a publication
<cjwatson> And "upload_distroseries" in processaccepted is actually SPPH.distroseries, not .upload_distroseries
<cjwatson> The naming lies
<wgrant> So keeping the upload_blah in queue.py wins for everyone
<wgrant> Yeah, because when the function was originally written it was always the upload distroseries.
<cjwatson> So yes, that can and should be non-nullable.
<wgrant> Great
<wgrant> Limiting the propagation of mistakes from 2005 is good :)
<wgrant> I shall give you a number.
#launchpad-dev 2012-08-05
<ewanj> hello
<ewanj> hi all, I'm looking at applying for the canonical software engineer vacancy but thought I'd try and find out a little more about it. How long have people been working with launchpad? What projects have you worked on? What is it like working remotely (i.e. do you still have the standard 9-5 days or, because everyone is distributed worldwide, do you find yourself working obscure hours)?
<ewanj> Thank in advance everyone
<lifeless> 7 years, lots, great, and routine hours are important.
<ewanj> Thanks lifeless
#launchpad-dev 2013-07-29
<cjwatson> wgrant: https://code.launchpad.net/~cjwatson/launchpad/buildstatus-aborted/+merge/176990 does some more useful things with BuildStatus.ABORTED now, although it's also noticeably more complex.
#launchpad-dev 2013-07-30
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/destroy-pppd/+merge/172945
<wgrant> StevenK: Why does getPendingDiffs still exist in the model?
<StevenK> That's a good point
<StevenK> Only tests call it
<StevenK> wgrant: I shall destroy it with fire
<cjwatson> I believe I have all the build cancellation code written and proposed-for-merge now.
<cjwatson> (Rather tempted to convert large swathes of buildmaster to defer.inlineCallbacks, but maybe not just now ...
<cjwatson> )
<cjwatson> wgrant: Did we decide what we wanted to happen wrt the devel alias if a PPA once had publications in a later series, but then they are deleted and only publications in earlier series remain?
<cjwatson> wgrant: Looking at the likely complexity of the query for "latest series with any active source or binary publications", I'm at least half-tempted to just do it as an existence check on the filesystem when creating the symlinks; but that would only work if the answer to that question is "leave the alias pointing to the most recent series anyway".
 * cjwatson nukes the easy tag on bug 54730.  It wasn't. :-P
<_mup_> Bug #54730: launchpad buildd abort does not work as expected <launchpad-buildd:In Progress by cjwatson> <https://launchpad.net/bugs/54730>
#launchpad-dev 2013-07-31
<wgrant> cjwatson: Hm, latest by DS.id should be quick and easy, but latest by DS.version is not. I guess that latest (non-zero size?) on disk would work
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/destroy-pppd/+merge/172945 , now with far more destruction
<StevenK> wgrant: And thanks for the prod -- that branch pulls PDJ and its infra down to -2 from +104
<wgrant> Will look at reviews in a sec
<wgrant> Been vaguely up since 4am, so not entirely alive all day
 * StevenK peers at wgrant 
#launchpad-dev 2013-08-01
 * StevenK peers at this doctest
<StevenK>     >>> transaction.commit()
<StevenK>     >>> flush_database_updates()
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/destroy-some-sampledata-constants/+merge/177977
<wgrant> StevenK: k
<stub> So rather than having the Librarian write to Swift when uploading a file, I was thinking I wouldn't modify that code path at all and always write to disk. The garbage collector processes would be responsible for pushing files into Swift. That way, the only modification to the twisted daemon is to attempt to stream the file from Swift, and if that fails, fallback to streaming from disk.
<stub> This also writes the migration-of-existing-files code - two birds with one piano
<stub> wgrant, StevenK : How does that sound?
<cjwatson> wgrant: https://code.launchpad.net/~cjwatson/launchpad/series-alias/+merge/178103
#launchpad-dev 2014-07-28
<lifeless> wgrant: what things are different in the new microservices plan that you're hiring for vs my sketch
<wgrant> lifeless: We're slicing in both directions. Moving the core of Launchpad toward being more of a platform providing infrastructure for services (like Bugs, Code, Translations and Soyuz) to be built upon, and a portal pulling data from those services together.
<wgrant> Splitting out the backend services is a start, but it doesn't solve the real big problems with development and maintainability.
<wgrant> We need engineering-inclined translators to be able to work on a 50KLOC Translations codebase without worrying about the 100KLOC of Soyuz that's intertwined with it.
<wgrant> And Translations needs to be able to have terrible database behaviour without impacting Soyuz.
<wgrant> etc.
<lifeless> so specifically siloed data stores?
<wgrant> We can provide better user-visible integration between applications than we have today, but with the applications running as separate services.
<wgrant> Yes.
<lifeless> interesting, I'll be keen to hear how it turns out
<lifeless> do you think you'll do some of the core facilities (e.g. the notification service) along the way ?
<wgrant> Definitely.
<wgrant> We need some core backend bits microservicified before we can split out most of the apps.
#launchpad-dev 2014-07-29
<wgrant> cprov: Could you please have a look at https://code.launchpad.net/~wgrant/launchpad/nu-overrides-adapters/+merge/228619?
<cprov> wgrant: yes, of course
<cprov> wgrant: it means that now we can support PPA uploads to universe and properly override them ?
<wgrant> cprov: PPAs deliberately override everything to main.
<cprov> wgrant: I know, but is it still deliberately rejecting uploads targeted to other components.
<wgrant> cprov: Ah, no, that was fixed in like 2008 :P
<wgrant> But in an awful way that I can just about remove now.
<wgrant> The methods that create SPPHs and BPPHs have an "if archive.is_ppa: component = main" hack.
<cprov> wgrant: ah, it's in *PPHs ... I could not find it.
<wgrant> Specifically, the get_component function in lp.soyuz.model.publishing
<cprov> wgrant: it seems too hairy for changing in this MP.
<wgrant> cprov: You'll see that I left XXXs in place in find_and_apply_overrides. Since archiveuploader applies overrides by mutating BPR and SPR, and we want to remember whether a PPA package had contrib or non-free in its Section field, we can't apply PPA overrides in NascentUpload -- we have to maintain the publish-time hack instead.
<wgrant> But eventually PackageUpload will store overrides directly, and we can stop mutating the SPR and BPRs, and those hacks can go away.
<cprov> wgrant: agreed
<wgrant> cprov: Thanks.
 * wgrant breaks the world.
#launchpad-dev 2014-07-30
<wgrant> stub: Shall we deploy the first phase of the DB patch tonight?
<stub> wgrant: yup
#launchpad-dev 2014-07-31
<cjwatson> jelmer: Would you mind having a quick look at https://bugs.launchpad.net/launchpad-buildd/+bug/1350430 ?  bzr-builder doesn't call substitute_changelog_vars for format 0.4; it looks deliberate, but I can't figure out why, nor what's supposed to be done instead
<_mup_> Bug #1350430: {debupstream} {debversion} not recognised  by format 0.4 <launchpad-buildd:New> <https://launchpad.net/bugs/1350430>
<cjwatson> jelmer: (we've finally deployed a non-archaic bzr-builder, so just starting to notice the effects of things you did years ago ...)
<StevenK> OMG, so it ain't so
<cjwatson> StevenK: scalingstack, it is a thing at last
<StevenK> I don't believe it
<cjwatson> jelmer: http://paste.ubuntu.com/7910544/ seems to fix it, and the unit tests still pass; but that line of code seems to have been added after the 0.4 format was added if I'm reading the history right, so it would be nice to get confirmation before pushing that to utopic/trusty-updates and potentially breaking something else ...
<cjwatson> __init__.py says "{debupstream} now only looks for changelog in the root branch, not the resulting tree" - struggling to work out why that's a useful thing
<cjwatson> the bug suggests {debversion} is affected too, which makes sense given the structure of substitute_changelog_vars, so presumably also {debupstream-base}
<jelmer> cjwatson: looking
<jelmer> it's a long time I've worked on all of this :)
<cjwatson> yeah :)
<jelmer> cjwatson: I vaguely remember that the point was that we could find out the version string without actually building the tree
<jelmer> there was also a syntax to get the debversion from a nested tree
<jelmer> {debversion:NAME}
#launchpad-dev 2015-07-27
<mwhudson> wgrant: general rebuild testing question
<mwhudson> i did a rebuild test and millions of builds failed
<mwhudson> once i've fixed the problems, is it easiest just to make a new ppa and start again?
<wgrant> mwhudson: If the builds failed, you can just retry them.
<wgrant> But you can't retry a build that was successful but wrong.
<wgrant> You need to either upload a new version of the successful packages, or create a new PPA.
<mwhudson> i guess there is a script somewhere to retry all failures?
<mwhudson> i don't think i have any successful but wrongs this time
<wgrant> Something like "for build in lp.archives.getByReference(reference='~some/ppa/path').getBuildRecords(build_status='Failed to build'): build.retry()"
<wgrant> I don't know if there's an existing script for it.
<mwhudson> cool thanks
 * mwhudson gets back to installing headers somewhere builds can find them...
<blr> wgrant: collander schemas are kind of nice, might be worth adding better validation to turnip's api using that at some point.
<blr> err colander
<wgrant> I've not looked at colander in depth.
<blr> it lets you provide a validation schema for an endpoint and handles error messages appropriately
<blr> morning
<mwhudson> ppas for debian pls
#launchpad-dev 2015-07-28
<mwhudson> blr, wgrant: woo the (somewhat) despecial casing of launchpad in the go tool just landed
<wgrant> Yay
<blr> mwhudson: nice, progress of sorts!
<mwhudson> yeah
<mwhudson> the last two days' worth of fun is finding out what breaks in the archive with go 1.5...
<mwhudson> vs what's broken already
<wgrant> Everything and most things?
<wgrant> Or, on x86, more than everything.
<wgrant> s/x86/!x86/
<mwhudson> heh well, poking as few bears as possible
<mwhudson> so not building 1.5 for arm64 or ppc64le yet
<mwhudson> or using shared libs
<mwhudson> 20 packages have failures, about half of which are not my fault
<mwhudson> (already broken with gccgo in archive, or ftbfs on any arch for some reason)
<wgrant> Ew
<mwhudson> https://docs.google.com/spreadsheets/d/1hsw1qAy0yLAfqUnZ9fVbXKwe7HxwUzpx5kJ72bF6nUE/edit#gid=0
<mwhudson> wgrant: btw, chdist is super handy, thanks for reminding me about that
<wgrant> :)
<wgrant> cjwatson: Is --check-distribution useful given the --check-archive option?
<wgrant> eg. ubuntu-archive-tools always infers distribution from archive.
<cjwatson> u-a-t provides distribution options too in case you didn't want to specify an archive ...
<wgrant> Oh, so it does.
<wgrant> Though in this case it's --check-distribution=ubuntu in place of --archive=ubuntu, which doesn't seem like much of a win.
<cjwatson> I dunno, I was mostly being regular
<cjwatson> Can remove them if you prefer
<cjwatson> s/them/it/
<wgrant> I'm not fussed, but it doesn't seem particularly useful.
<cjwatson> Inferring distribution from archive makes sense at least; I'd forgotten that u-a-t did that.
<wgrant> The non-archive methods only exist on the API for compatibility reasons and for partner.
<wgrant> In u-a-t it's probably just compatibility.
<wgrant> -A ubuntu vs -d ubuntu is hardly an ease of use problem.
<cjwatson> Depends on the tool; in e.g. manage-chroot it makes more sense to talk about the distribution.
<wgrant> Indeed.
<wgrant> But it's not operating on an archive at all in that case (potentially a misfeature, as we've run into a few times...)
<cjwatson> wgrant: updated MP, how about now?
<wgrant> cjwatson: Looks good, thanks.
<cjwatson> wgrant: as for https://code.launchpad.net/~cjwatson/launchpad/db-livefs-build-ordering/+merge/265660, the existing index becomes inoperative anyway once the new code is deployed.  Since it's a smallish table I decided not to worry too much
<wgrant> cjwatson: Ah, fair point.
<wgrant> cjwatson: My main concern was the long-running transaction thing, anyway.
<wgrant> Don't want to be stuck for a 12 hour backup.
<cjwatson> Remind me of the backup schedule?
<cjwatson> Maybe we should just FDT it shortly after the code update.
<wgrant> Nah, live is fine.
<cjwatson> I was planning to request the whole thing once sourcherry gets round to applying this one.
<cjwatson> Did you get a chance to look at any of my snap branch stack?
<wgrant> Was unwell and off yesterday, and not 100% today, so I've merely pondered them with no obvious objections.
<cjwatson> Ah, sorry to hear that.
<cjwatson> I'm still slightly recovering from camping in a field for three days, but my inbox has more than enough to keep me occupied?
<cjwatson> s/\?/!/
<wgrant> Oh dear, a field!
<cjwatson> Distressingly outdoors.
<wgrant> The weather looked nice, though.
<cjwatson> Uh
<wgrant> Well, the temperature, at least.
<wgrant> I didn't check the rest :)
<cjwatson> We got pretty wet on Sunday.  Traditional festival weather.
<cjwatson> Good fun though
<wgrant> Excellent.
<cjwatson> wgrant: Hmm, I wonder what to do about stuff like https://launchpad.net/ubuntu/+source/gem2deb/0.20.1/+build/7722524.  It's beginning to look like we might be best off copying the entirety of Dpkg::Deps into our backported sbuild package as Sbuild::Deps
<cjwatson> (an approach previously taken in Debian buildd branches ...)
<wgrant> cjwatson: Hum, what happened there? It just excluded anything with a profile restriction, even when it was a negative?
<cjwatson>   DB<2> x deps_parse("ruby-rspec <!nocheck>", reduce_arch => 1, host_arch => "i386", build_arch => "i386", build_dep => 1, reduce_profiles => 1, build_profiles => [])
<cjwatson> 0  Dpkg::Deps::AND=HASH(0x9286abc)
<cjwatson>    'list' => ARRAY(0x90fb1e4)
<cjwatson>         empty array
<cjwatson> ^- that happened
<cjwatson> I guess that's a long way of saying "yes"
<wgrant> Quite.
<wgrant> How unfortunate.
<cjwatson> I don't know why there were no warnings or anything
<cjwatson> Dpkg::Deps is probably fairly self-contained on top of an otherwise modernish libdpkg-perl.
<cjwatson> I think it just needs Dpkg::BuildProfiles (possibly only on precise).
<cjwatson> In fact on both precise and trusty, since trusty's doesn't have parse_build_profiles or evaluate_restriction_formula.
<cjwatson> Needs Dpkg::BuildEnv on precise, but I think that's the lot.
<cjwatson> wgrant: Proposed sbuild patches: http://paste.ubuntu.com/11954598/ (precise) http://paste.ubuntu.com/11954600/ (trusty)
<rpadovani> cjwatson, o/ how are you? :-)
<cjwatson> wgrant: hmm, https://launchpad.net/ubuntu/+archive/test-rebuild-20150722/+build/7682014 looks weird - it's not a non-UTF-8 issue, because the byte reported is the first byte of Ã¶ in UTF-8
<cjwatson> but it seems astonishing that we would still have bugs with UTF-8 .changes
<wgrant> cjwatson: Oh, interesting.
<wgrant> cjwatson: I saw a couple of instances of that in the OOPS reports a week or so ago.
<wgrant> But it wasn't widespread, so I assumed it was legitimate badness.
<wgrant> It's presumably the maintainer email fixing changes failing to decode it somewhere.
<cjwatson> Regression from r17611, perhaps
<wgrant> Yeah, must be.
<wgrant> I'll try to repro.
<cjwatson> Thanks
<cjwatson> I remember there being functions in there that had decoding as kind of a side-effect, and thought I'd managed to disentangle all of that but maybe not totally correctly.
<wgrant> Ah, it wasn't in OOPS reports, it was in process-upload.py cronspam.
<wgrant> https://pastebin.canonical.com/136298/
<wgrant> Right, reproducible easily.
<wgrant> Will fix.
<wgrant> Yay SQLObject compat layer.
<cjwatson> Well
<cjwatson> It's not really sqlobject's fault
<wgrant> (test suite would have caught this if it were native Storm)
<cjwatson> Ah, OK
<wgrant> Because Storm whines if you try to put a str in a unicode column.
<wgrant> It's much more Python 3 about things.
<cjwatson> That value would previously have been ascii_smashed, so still a str but without any non-ASCII
<cjwatson> (And a lot less useful)
<wgrant> Very annoying for whipping up code quickly, very handy for not writing dodgy code.
<cjwatson> I made it be str for compatibility, but maybe it would have been better to fix all the callers to accept unicode
<wgrant> ALEFLAMMEIFORGETTHEREST
<cjwatson> Exactly
<cjwatson> Semantics of that value are "the RFC822-encoded version of this field", so you can probably make a case for either octet-stream or text
<cjwatson> Oh in fact it's just the name isn't it
<cjwatson>     name = force_to_utf8(name)
<cjwatson> So I guess just .decode
<wgrant> really?
<cjwatson> All very silly code.
<wgrant> what on earth
<wgrant> have you *read* force_to_utf8?
<cjwatson> I have.
<wgrant> That is certainly the stupidest thing I've seen today.
<cjwatson> It probably sort of made sense with the prevailing common case of pre-UTF-8 control files.
<cjwatson> Kind of.
<wgrant> No
<wgrant> Read it
<wgrant> Oh
<wgrant> I misread.
<cjwatson> Yes, I know :)
<wgrant> Oops
<cjwatson> ISO-8859-1 was probably the most common non-UTF-8 encoding for those though
<wgrant> I misread it as doing something with the decoded UTF-8 value
<wgrant> So it returned a unicode in one case and not the other.
<wgrant> Yeah
<wgrant> Hmmm
<cjwatson> It should arguably be turned into force_to_unicode
<cjwatson> If that doesn't make all the twisty callers explode too badly, and we already know our test coverage is poor ...
<cjwatson> Anyway, I must to bed
<wgrant> Night.
#launchpad-dev 2015-07-29
<rpadovani> cjwatson, o/ :-)
<cjwatson> rpadovani: Sorry, please don't just do that, say what you want.
<cjwatson> rpadovani: (I'm busy debugging home network problems, unlikely to be immediately available, but if you actually say what you want then I can get back to you later)
<rpadovani> cjwatson, oh, okay, sorry, didn't want to bother you - just asking if you had any change to review my branch - only because I'll not be available for next two weeks, so if there is something to fix I would like to do this week - but don't worry, take your time :-)
<wgrant> cjwatson: Thanks for all the reviews.
<cjwatson> gotta do something while waiting for my ISP to sort out my fault ...
<wgrant> Oh dear, what's happened?
<cjwatson> router lost its mind, had to factory reset, autoprovisioning didn't work properly.  fixed now.
<wgrant> Ah, unhelpful.
<Philip5> i just uploaded to my ppa on launchpad and got the files rejected with "utopic is obsolete and will not accept new uploads." what's this about? have i missed something or is it a bug?
<blr> morning
<blr> Philip5: Utopic's EOL was July 23rd
<Philip5> blr: noticed that now and missed the date. also thought launchpad would support builds longer than that but i understand, thanks
<blr> Philip5: there may be exceptional cases where we do, I'm not certain. wgrant will be able to tell you more in a few hours.
<Philip5> oki
<cjwatson> We don't generally support builds past Ubuntu's EOL, no.
<wgrant> It's pretty essential that people not run EOL Ubuntu releases -- security vulnerabilities are a thing.
#launchpad-dev 2015-07-30
<blr> hmm ended up ditching colander, validation isn't really complex enough to justify another dep, but worth looking at for larger services.
<blr> wgrant: token invalidation all sorted, was thinking we'd run the sweeper every minute?
<blr> would mean a token can potentially live 1m longer than it's ttl, although I would imagine most will be manually invalidated
<wgrant> blr: I don't think it's particularly necessary to run it that frequently. Presumably the thing validating a token would check the TTL before allowing it.
<blr> wgrant: it does, but what about the case where a token is manually invalidated and still within the ttl?
<blr> I suppose I should check both
<wgrant> Right, the pruning cronjob is just to clean up the DB.
<blr> oh, I am checking both :P
<blr> will add the cron job and schema migration to the charm tomorrow, then should be a in a reasonable state.
<blr> I seem to be leaking 'a's... time for coffee.
<wgrant> Heh
<wgrant> Sounds good
<cjwatson> wgrant: cross-ownership deletion constraints: we have those on recipes too, right?
<wgrant> cjwatson: yes, and they're horrible.
<wgrant> These are marginally worse (as they involve archives too), but not substantially.
<cjwatson> I'd never noticed them before.  I assume it means that you can prevent somebody else from deleting their branch by creating a recipe for it
<wgrant> Right, exactly.
<wgrant> There are a couple of bugs about it.
<cjwatson> How about making them nullable in this case?  You wouldn't be able to issue any new builds, but the existing snap builds could stick around/.
<wgrant> Right, that might be reasonable.
<wgrant> It is useful to compare the FKs to simple URLs from the snapcraft.yaml
<wgrant> Only the root branch actually has the FK enforced.
<wgrant> Because the rest are in freeform text from Launchpad's PoV.
<cjwatson> Freeform but parsed, but yes.
<wgrant> Not from Launchpad's PoV.
<cjwatson> SourcePackageRecipeDataInstruction
<cjwatson> Unless you're talking about snaps.
<wgrant> Oh, yeah, I mean snaps.
<wgrant> We can't enforce that references in snapcraft.yaml remain good.
<wgrant> So perhaps we need not enforce that the reference *to* snapcraft.yaml remains good.
<cjwatson> Right, hence why staleness doesn't work.
<wgrant> One could argue that the branch/repo and the archive are just references that may become stale like any other.
<wgrant> In fact the git_path has to be.
<cjwatson> Indeed.
<cjwatson> I'll weaken the constraint.
<wgrant> It is similar to what we do with SPRB.recipe.
<cjwatson> Weird place to break the link.
<cjwatson> I definitely prefer having SnapBuild.snap non-nullable.
<wgrant> For Snaps it is simpler, as the SnapBuilds are likely themselves deletable.
<wgrant> It was not so with SourcePackageRecipeBuilds.
<wgrant> Well, it would have been possible, but important authorship information would also have been lost.
<wgrant> For packages in archives.
<cjwatson> True
<cjwatson> Also one of these days I shall remember that unique constraints are indexes.
<wgrant> Indeed they are.
<wgrant> cjwatson: Have you thought about how private branches/repos and snaps might work?
<cjwatson> Not yet.  And hm, I'm missing information_type on snap, aren't I.
<cjwatson> I think that's the only piece that requires consideration at the DB level
<wgrant> It is conceivable.
<cjwatson> Other than that it's mostly granting some kind of token to the builder
<cjwatson> And I do need to handle processors as well; not just for restrictedness, but also just to configure which architectures to build for.
<wgrant> That is true, if they're not going to be explicitly requested.
<cjwatson> I guess we don't have daily builds yet so it could be entirely a matter of explicit requests, but that seems like poor UX.
<cjwatson> Would be nicer to record which arches are currently being built and then have a button to build everything now pls.
<cjwatson> wgrant: Perhaps SnapArch would be a reasonable thing to have?  I'm thinking it could pretty much exactly duplicate Archive.processors etc.
<wgrant> cjwatson: That's certainly what I'd do.
<cjwatson> Maybe without the enabled_restricted_processors stuff.
<wgrant> Right, it doesn't particularly need the backward compat bits.
<cjwatson> I mean, we want to avoid letting people set restricted processors, but it doesn't need that name.
<wgrant> Oh, right.
<wgrant> It may be a good opportunity to fix it for archives too.
<cjwatson> I don't want to bite off any more just right now, my mouth is full already :P
<wgrant> Damn.
<cjwatson> Is there any point in having SnapArch.id, given that it'd be UNIQUE (snap, processor) anyway?
<wgrant> No.
<wgrant> ArchiveArch probably dates from not long after the Storm transition.
<cjwatson> wgrant: OK.  Do my changes to https://code.launchpad.net/~cjwatson/launchpad/db-snappy/+merge/265332 look OK?
<cjwatson> Updating the model code now
<wgrant> 5+public.snaparch                         = SELECT, UPDATE
<wgrant> orlyu
<cjwatson> Hm yeah that's probably wrong
<cjwatson> Nothing other than launchpad_main should need UPDATE
<wgrant> Nothing should ever be updating them, should it?
<cjwatson> Fair point.  Both fixed.
<wgrant> I think that's reasonable then. Thanks.
<cjwatson> I'll add and test Snap.processors before landing.
<cjwatson> Oh, information_type, I forgot that too.
<cjwatson> wgrant: If you're still up, do you think I should have Snap.information_type (like proper shareable things) or just Snap.private (like Archive)?  Maybe I should defer privacy handling until a bit later, or I'll never get this going
<wgrant> cjwatson: I would prefer Snap.private, but I have long had a potentially unreasonable vendetta against information_type, so take it with a grain of salt.
<wgrant> I see no benefit here; we'll be restricting it to two values anyway...
<cjwatson> I guess if it's just private I can probably get that into the first pass.
<wgrant> The main difficulty with privacy is working out who can see it.
<wgrant> Since these things aren't within another privacy context (ignoring the team).
<cjwatson> Right, that's why I was sort of inclined to defer to start with
<wgrant> I cannot disagree.
<wgrant> It is not significantly more difficult to retrofit.
<cjwatson> And it's a bit multifaceted - do we hide the snap if the branch is invisible?
<wgrant> Oh it is terrible, yes.
<cjwatson> Though I suppose that just goes on the security adapter
<cjwatson> But still
<wgrant> We have the snap, the source, and the archive.
<wgrant> All of which may be differently private.
<wgrant> It is quite ghastly.
<cjwatson> Anyway, I think I've addressed all your review comments now, so landing db-snappy
<cjwatson> Let's see if I remembered everything this time
<cjwatson> Oh, damnit, missing ON DELETE CASCADE
<wgrant> cjwatson: Where were you going to use that?
<cjwatson> SnapArch.snap
<wgrant> Ah, I suppose that makes sense.
<cjwatson> How do I add that after the fact?  ALTER TABLE SnapArch DROP CONSTRAINT snaparch_arch_fkey; ALTER TABLE SnapArch ADD FOREIGN KEY (snap) REFERENCES snap ON DELETE CASCADE?
<wgrant> FKs aren't constraints.
<cjwatson> Seems annoying not to be able to alter that in one step.
<cjwatson> They kind of are.  \d lists them as such.
<cjwatson> Also no ALTER TABLE ... DROP FOREIGN KEY
<wgrant> Oh, that's right. They appear as constraints to some DML.
<wgrant> I'm not sure I'd bother, anyway.
<wgrant> There are other uncascadable FKs to Snap, so it needs a whole lot of custom deletion code anyawy.
<cjwatson> I guess.  Just noticed when I merged up as far as snap-builds and tests started failing.
<cjwatson> Its custom deletion code currently only consists of checking that there are no builds and then store.remove
<cjwatson> With that check there are no other uncascadable FKs.
<wgrant> No uncascadable FKs except the one that will affect anything that's actually been used, indeed :)
<cjwatson> I suppose in future we might want to be able to delete those.  It makes a bit more sense than it would for recipes where they'd have to cascade into package builds and obviously can't.
<wgrant> Right, exactly.
<cjwatson> Snap.requestBuild is going to need some fixing up to honour processors, and maybe we want Snap.requestBuilds
<blr> morning
#launchpad-dev 2015-07-31
<mwhudson> wgrant: hi, could i get https://launchpad.net/~mwhudson/+archive/ubuntu/gccgo-arm64-fixes enabled for arm64 please?
<wgrant> mwhudson: It is done (nonvirt + arm64)
<mwhudson> wgrant: thanks
<mwhudson> now to remember how the gcc packaging works
<mwhudson> there's something about the gcc packaging that turns me into an idiot
<mwhudson> three screwed up builds can take out most of a day :(
<mwhudson> wgrant: can i retry (through the api) a non-failed build?
<wgrant> mwhudson: No.
<mwhudson> ok
<mwhudson> i might start with a new go 1.5 rebuild test ppa on monday then
<mwhudson> (and try native go on arm64 and ppc64el for laughs)
<mwhudson> by when the gcc 5 as default thing will have happened, which is somewhat relevant to a few things too
 * mwhudson evaporates
#launchpad-dev 2015-08-02
<blr> morning
<mwhudson> wgrant: good morning, could you devirt and enable all arches except powerpc on https://launchpad.net/~mwhudson/+archive/ubuntu/go1.5-arch-tests please?
<wgrant> mwhudson: Why not powerpc?
<mwhudson> wgrant: i want to test the new support for arm64 & ppc64le in go
<wgrant> Oh, it doesn't support 32-bit BE?
<wgrant> How odd.
<mwhudson> right
<wgrant> mwhudson: Done.
<mwhudson> wgrant: ta
<blr> hi wgrant, how was your weekend?
<wgrant> ACPI is terrible.
<wgrant> Otherwise not bad.
<wgrant> You?
<blr> a little hectic, but good all the same.
#launchpad-dev 2016-08-02
<xnox> Does launchpad git support use https://github.com/franckverrot/git_fdw ?
<wgrant> xnox: No, that looks horrifying :P
<xnox> wgrant, thanks =) i needed a second opinion if that is amazing or terrible idea =)
#launchpad-dev 2018-08-01
<Guest50369> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<Guest50369> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<Guest50369> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<Guest50369> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<syncretism_mbl12> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<syncretism_mbl12> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<syncretism_mbl12> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<syncretism_mbl12> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<swarfega18> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<swarfega18> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<swarfega18> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<swarfega18> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<ProClifo> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<ProClifo> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<ProClifo> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<ProClifo> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<tx15> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<tx15> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<tx15> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<tx15> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<depleted> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<depleted> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<depleted> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<depleted> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<n0nada21> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<n0nada21> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<n0nada21> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<n0nada21> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<tigermousr3> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<tigermousr3> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<tigermousr3> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<tigermousr3> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<arza7> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<arza7> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<arza7> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<arza7> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<liguo> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<liguo> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<liguo> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<liguo> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<prettymuchbryce9> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<prettymuchbryce9> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<prettymuchbryce9> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<prettymuchbryce9> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<jlf23> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<jlf23> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<jlf23> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<jlf23> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<issyl015> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<issyl015> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<issyl015> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<issyl015> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<PrettyKittie7> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<PrettyKittie7> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<PrettyKittie7> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<PrettyKittie7> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<Turandot26> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<Turandot26> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<Turandot26> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<Turandot26> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<Vlad13> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<Vlad13> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<Vlad13> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<Vlad13> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<marcoslater> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<marcoslater> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<marcoslater> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<marcoslater> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<EvilWerezombie8> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<EvilWerezombie8> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<EvilWerezombie8> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<EvilWerezombie8> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<EdSaperia21> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<EdSaperia21> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<EdSaperia21> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<EdSaperia21> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<Yatekii20> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<Yatekii20> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<Yatekii20> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<Yatekii20> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<mcintosh1> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<mcintosh1> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<mcintosh1> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<mcintosh1> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<Gizmokid20053> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<Gizmokid20053> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<Gizmokid20053> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<Gizmokid20053> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<Bock> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<Bock> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<Bock> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<Bock> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<ChickeNES> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<ChickeNES> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<ChickeNES> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<ChickeNES> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<celyr13> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<celyr13> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<celyr13> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<celyr13> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<kepler_mach19> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<kepler_mach19> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<kepler_mach19> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<kepler_mach19> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<sam16> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<sam16> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<sam16> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<sam16> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<Pugabyte14> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<Pugabyte14> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<Pugabyte14> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<Pugabyte14> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<haza-w26> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<haza-w26> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<haza-w26> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<haza-w26> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<emerson> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<emerson> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<emerson> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<emerson> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<threeFifths> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<threeFifths> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<threeFifths> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<threeFifths> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<Turandot5> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<Turandot5> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<Turandot5> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<Turandot5> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<MalReynolds6> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<MalReynolds6> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<MalReynolds6> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<MalReynolds6> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<Unit193> cjwatson: Hi, I've set +r due to the spam above, at the request of xnox.
#launchpad-dev 2019-07-29
<SpecialK|Canon> Am I right in my understanding that https://dev.launchpad.net/Running is "running _the launchpad web app_" and not necessarily "all the other services that make up LP" (buildds and turnips etc.)?
<SpecialK|Canon> (Waiting for it to finish atm)
<wgrant> Right. There's HowToUseSoyuzLocally, HowToUseCodehostingLocally for the other bits
<wgrant> Or names along those lines
<cjwatson> I think that running turnip is only documented in its own README
