#launchpad-dev 2010-06-28
<thumper> morning
<thumper> those 4 hours difference is a killer
<thumper> I'll be a little late starting
<lifeless> :>
<lifeless> s/will be/am/ :>
<adeuring> good morning
<mrevell> Morning
<jml> morning
<lifeless> hi
<nigelb> morning jml :)
<deryck> Morning, all.
<bigjools> wgrant: hey, around?
<wgrant> bigjools: Hi.
<bigjools> wgrant: hey - can you confirm you fix for https://bugs.edge.launchpad.net/soyuz/+bug/592935 is ok?
<_mup_> Bug #592935: Disabled PPAs still shown on person index <qa-needstesting> <Soyuz:Fix Committed by wgrant> <https://launchpad.net/bugs/592935>
<bigjools> your*
<bigjools> it's a little harder for me to check as a PPA admin :)
<wgrant> bigjools: Done.
<bigjools> wgrant: cheers
* jml changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.06 | PQM is open | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
* henninge changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.06 | PQM is in Release-Critical mode; devel is open  | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<james_w> code import jobs don't use the job system?
<lifeless> Which one :P
<lifeless> AFAIK there are three 'do stuff' systems + many adhoc ones; the import one, the buildd one and the branch scanning/archive copying one.
<james_w> lp.services.job
<_thumper_> gah
<_thumper_> slept in again
<lifeless> lol
<lifeless> I'll happily take your extra sleeping hours
<lifeless> how does quassel know you're changing host ?
<thumper> now we have to do in 30m what we normally take 90 for to get the kids into school
 * thumper shrugs
<Ursinha> sinzui, don't worry about the mistagged bugs, I'm checking one by one and retagging them
<sinzui> Ursinha, I just used https://bugs.edge.launchpad.net/launchpad-project/+bugs?field.tag=qa-needstesting&orderby=targetname
 * sinzui is done tagging
<Ursinha> sinzui, right, just telling you you don't have to worry about retagging them :)
<rockstar> abentley, delete or destroySelf?
<abentley> rockstar, I'd go for destroySelf.
<rockstar> abentley, ack, thanks.
#launchpad-dev 2010-06-29
<poolie> sinzui, so did bug expiry actually get reenabled?
<poolie> post bug 333521
<_mup_> Bug #333521: Enable bugs expiration for Ubuntu <qa-ok> <Launchpad Bugs:Fix Released by adeuring> <https://launchpad.net/bugs/333521>
<wgrant> poolie: There has been no screaming, so I doubt it.
<wgrant> The plan fortunately seems to have vanished.
 * thumper needs more coffee
<lifeless> wgrant: what, you don't like bug expiration?
<wgrant> lifeless: Not the latest proposal.
<wgrant> of switching it on without unsetting the flag globally.
<adeuring> good morning
* henninge changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.06 | PQM is open  | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<mrevell> Morning
<jml> good morning
<deryck> Morning, all.
<adeuring> morning deryck
<bigjools> zope.configuration.config.ConfigurationExecutionError: <type 'exceptions.ValueError'>: /var/tmp/launchpad_mailqueue is not a Maildir folder
<bigjools> wtf
<bigjools> sigh, why does /var/tmp/launchpad_mailqueue get created with root ownership, yet "make" wants to remove it etc.
<bigjools> (as my local user)
<wgrant> It would be interesting and not too difficult to calculate the actual buildd utilisation at times like this.
<wgrant> I suspect it's well under half of the 100% that it appears to be.
<wgrant> It's a bit concerning that one can actually watch buildd-manager's progress through a single cycle over several minutes by refreshing /builders.
<bigjools> We know.  It's getting fixed.
<bigjools> wgrant: so still 261k where date_started is null and status != 0;
<bigjools> mmm
<wgrant> bigjools: Er, really?
<bigjools> really
<wgrant> But that is the same as the number you quoted without the filte.
<wgrant> And there are somewhere around 20k builds pending.
<wgrant> And probably tens of thousands more suspended.
<wgrant> So it does not compute.
<bigjools> well I am using staging's DB
<bigjools> which is a little out of date but near enough
<bigjools> oh, meh
<bigjools> we blitz all pending jobs on staging now
<bigjools> it has a buildd-manager
 * bigjools dodges the bird that just flew into the office
<wgrant> What if you say status NOT IN (0, 5)?
<wgrant> (5 being superseded)
<wgrant> staging has a build farm now? Handy.
<bigjools> 120k
<bigjools> well if you count one machine with two builders on it as a farm ...
<wgrant> Aha.
<wgrant> OH~!
<wgrant> gina.
<wgrant> 22:14 < wgrant> No date_started but successfully built is fine.
<wgrant> 22:14 < wgrant> No date_started, but successfully built *and with date_finished or builder* is not.
<wgrant> I got my issues from that day mixed up :/
<wgrant> There were 9 builds in this state at that time.
<wgrant> It should be the same now.
<bigjools> gror
<bigjools> hardware failures....awesome
<wgrant> Bird-related? :P
<bigjools> no, the machine I was just running queries on :/
<wgrant> Oh, excellent.
<bigjools> not exactly
<bigjools> wgrant: can you clarify that clause again?
<bigjools> oh nm
<jml> little bit distracted today, going to see what breaks when I run the tests with no branch sample data.
<bigjools> wgrant: http://pastebin.ubuntu.com/456906/
<bigjools> gnargh
<bigjools> oh nm there's only 9, stray clause table
<wgrant> Ah, good.
<wgrant> I was WTFing a bit there.
<bigjools> def just 9
<bigjools> ok, now, what shall I poke in the start date
 * bigjools ponders
<wgrant> The 'builder IS NOT NULL' disjunct was only required when there may have still been incomplete builds.
<wgrant> You could poke in something just around the rollout time.
<bigjools> yeah
<wgrant> Or if you're feeling adventurous actually grab the start time from the logs.
<wgrant> But that's probably pointless.
<EdwinGrubbs> henninge_: did you see my email about caching the message count?
<bigjools> wgrant: have you got any idea why someone would write that first line of code in IArchive.checkArchivePermission()
<bigjools> maybe they were on a low IQ day
<wgrant> bigjools: Ah, that's why it's OOPSing?
<bigjools> yes
<bigjools> words fail me
<wgrant> Well, it's not terribly unreasonable if one assumes it's only used for checking upload permissions.
<bigjools> it should just return False FFS
<wgrant> Well, if you're checking whether a source should be accepted into a copy archive, you probably want to crash.
<wgrant> But this method is used elsewhere, so...
<bigjools> no I never want to crash
<bigjools> returning False is entirely appropriate
<wgrant> If you get to the stage where you are verifying permissions for a copy archive, you probably want to let someone know.
<wgrant> Because something is seriously broken.
<bigjools> there's too much of that paranoia in Soyuz
<henninge> EdwinGrubbs: yes I saw it but have not read it yet.
<henninge> sorry ;)
<wgrant> bigjools: I think it's a handy defensive tactic, given how badly things can go wrong.
<wgrant> Just... not when applied incorrectly.
<bigjools> the latter is my beef
<wgrant> That line has been there for a while.
<wgrant> i wonder why it hasn't been problematic before.
<wgrant> I guess maybe canUpload wasn't actually used for that before :/
<wgrant> It is a reasonably novel method.
<bigjools> it's used when ascertaining retry availability
<bigjools> if you can upload you can retry
<wgrant> I know why it's used now.
<wgrant> I'm just wondering why it never appeared before, when that code has been there for ages.
<wgrant> And those pages worked a few months ago.
<bigjools> you have to hit the build in a certain state
<maxb> <bigjools> if you can upload you can retry <--- I can upload to ~launchpad/+archive/ppa but I can't retry
<bigjools> maxb: it only applies to the main archive :)
<maxb> consistency? What's that? :-)
<bigjools> maybe it should not
<wgrant> I've been meaning to fix that for copy archives for a while.
<wgrant> I didn't realise it affected PPAs too.
<bigjools> well for copy archives it should only let the owner and admins
<wgrant> Or custom uploaders, if they are added.
<wgrant> No reason to forbid it, and it's handy.
<bigjools> custom uploaders are sorta evil at the moment
<wgrant> eg. MOTU should be able to retry builds in the rebuild archive, without having to poke doko.
<bigjools> I want to move all upload permissions to explicit ArchivePermissions rather than relying on the ownership of the PPA
<wgrant> Yes yes yes.
<wgrant> All this implicit permission stuff is silly.
<wgrant> Although I guess we won't see those ACLs for a while now.
<cakofony> Does the launchpad api have a sleep type thing built in, or am I going to get banned from the site for using the api too much?
<bigjools> cakofony: not unless we can see it grossly affecting performance
<cakofony> bigjools: I'm working on an opensource project that collects as much data as possible from forge sites -- we pull a lot of data :-s
<bigjools> cakofony: is that a one-off or ongoing?
<cakofony> bigjools: we do a run every month or so (it probably takes a day or two)
<cakofony> bigjools: before I found the api, I was downloading about 100,000 html pages per run, this should speed things up quite nicely
<bigjools> okay - I might put you on to someone else like, errr, mrevell perhaps
<bigjools> and jml might be interested too
<jml> what
<jml> cakofony, there's no built in sleep
<jml> cakofony, if you become a problem, we'll let you know :)
<jml> cakofony, what's the website?
<cakofony> jml: flossmole.org
<jml> cakofony, thanks.
<cakofony> jml: thank you!  I'm calling the client in my program "python-flossmole"
<jml> cakofony, are you syncing bug data, or just projects & developers?
<cakofony> jml: only projects teams and developers
<Z-RAY_> after amateur tries to update MLT to 0.5.6 i have left without ffmpeg modules and even ffpmeg is installed, kdenlive says that some not installed at all. also it says that some sound module is not installed. i spent all day to make "lines and dots" bug dissappear (white lines and dots - was promised to be fixed in MLT 0.5.5) and i couldn't make it, even worse - now modules "avformat module", "Quimage module", "Title module" are missing and reinstallin
<Z-RAY_> g of the program and ffmpeg does not helping.
<Z-RAY_> help me please to make this thing work correctly. my skype is "woanerges", or write me here. please, bro's, come on, i need some support here!
<Z-RAY_> white dots and lines examples:
<Z-RAY_> http://kdenlive.org/sites/default/files/shot1_0.png
<Z-RAY_> http://www.youtube.com/watch?v=nrFXr_bx2a0
<rockstar> abentley, would you object if I factored out the sourcepackagerecipebuild views in browser sourcepackagerecipebuild.py ?
<abentley> rockstar, I don't understand.
<rockstar> abentley, so lp.code.browser.sourcepackagerecipe has views for sourcepackagerecipebuilds in them.  I want to move them out.
<abentley> rockstar, alright.
<rockstar> abentley, cool, thanks.
<thumper> morning
<lifeless> hiya thumper
<thumper> hi lifeless
#launchpad-dev 2010-06-30
<thumper> noticed we just passed bug 600000
<_mup_> Bug #600000: missing dependency on Bazaar <hitchhiker (Ubuntu):New> <https://launchpad.net/bugs/600000>
<lifeless> \o/
<spm> is that 600,000 bzr bugs? or am I being a transparent troll again?
<thumper> escaping the vortex of home to get some fud
<poolie> lifeless:  https://bugs.edge.launchpad.net/launchpad-foundations/+bug/600081
<_mup_> Bug #600081: count/time of for queries and page rendering should be visible, at least to developers <performance> <Launchpad Foundations:New> <https://launchpad.net/bugs/600081>
<poolie> what's even better is i can probably fix bug 600000
<_mup_> Bug #600000: missing dependency on Bazaar <hitchhiker (Ubuntu):New> <https://launchpad.net/bugs/600000>
<lifeless> poolie: cool, thanks for filing that - really good observation that it should be more visible.
<poolie> <flash>
<poolie> :)
<lifeless> We can do that just for mbp, if you like.
<poolie> giant fist of doom for everyone else
<lifeless> you are in a web site. there is a performance meter near the top. To the left, some content, to the right a menu. Don't look down.
<poolie> and we now have more bugs than bugs.d.o
<poolie> despite starting ~8 years later?
<lifeless> more users of ubuntu, and more users of lp
<poolie> and it's easier to file bugs, for better or worse
<adeuring> good morning
<thumper> poolie: reply sent to launchpad-dev list
<jml> thumper, ever wanted to know what tests would fail if we had no branch sample data? http://paste.ubuntu.com/457259/
<lifeless> jml: that is cool. I only saw your quesiton; sadly the operational 'work' part of network isn't so much, for me right now.
<lifeless> jml: its a lot less than it was once, perhaps ?
<jml> lifeless, I don't know.
<jml> lifeless, probably.
<jml> lifeless, I get the impression that the code team hasn't been adding tests which require sampledata, which counts for a lot.
<lifeless> I'm glad you did that experiment.
<lifeless> I wonder if its worth putting that in a cron job with an auto merge of trunk and an alert if it increases?
<jml> hmm.
<lifeless> probably not :)
<jml> I guess I think I could probably fix all of the tests and delete the branch sample data in only a few hours
<jml> the cronjob might take less time, but would be more fiddly
<lifeless> 82 tests
<jml> yeah. don't forget that the doctests are deceptively big.
<lifeless> if it was 3 minutes work per test, then I'd agree with that assessment
<lifeless> right
<lifeless> so I don't think its a few hours work.
<lifeless> < 1 week, > a day
<jml> :)
<lifeless> assuming no surprises
<lifeless> it would be worth doing a few hours work on it, if you have a few hours up your sleeve.
<jml> I might soon.
<lifeless> cool
<lifeless> hmm, back to my slides I think
<jml> me also.
<jml> good bye IRC.
<deryck> Morning, all.
<jtv> henninge: been looking at the pottery bug with quoted names in the gettext package name.  My solution is to have ConfigFile.getVariable strip quotes (at least if it's clearly a case of a properly-quoted string).
<henninge> jtv: sorry, got distracted.
<henninge> jtv: yes, that seems to be simple stripping of quotes.
<henninge> jtv: I was thinking of another bug that is more complicated.
<jtv> henninge: ah
<jtv> this is bug 580337
<_mup_> Bug #580337: pottery fails with quoted GETTEXT_PACKAGE <pottery> <Launchpad Translations:In Progress by jtv> <https://launchpad.net/bugs/580337>
<henninge> yes, I was thinking of bug 582185
<_mup_> Bug #582185: pottery produces "@PACKAGE@.pot" for lp:gawk <pottery> <Launchpad Translations:Triaged> <https://launchpad.net/bugs/582185>
<henninge> jtv: thanks for going after that fix.
<jtv> henninge: do you think I should strip [m4 quotes] as well?
<jtv> Or just single/double quotes?
<henninge> jtv: I thought I was already doing that.
<jtv> Yes, you are...  so not sure it has any value.
<jtv> I mean, not sure it would have any value for me to add that to getVariable.
<henninge> jtv: it would, if you remove it from whereever it is done now.
<jtv> It's definitely not limited to this particular variable... so would you like me to strip [bracket quotes] off as well?
<henninge> let me take a closer look
<henninge> jtv: so, I'd create a function/method that does the stripping and apply it to both the return values of ConfigFile.getVariable and .getFuncitonParams.
<jtv> Sounds sensible.
<jtv> OK, I can do that.
<henninge> jtv: then remove the stripping fom _get_AC_PACKAGE_NAME
<jtv> Yu
<jtv> p
<henninge> jtv: thanks
<henninge> ;-)
<jtv> but first, OCR.  :)  I need lunch & sunlight too.
<salgado> flacoste, I remember somebody saying that zope.deferredimport wouldn't help us workaround our circular dependencies.  do you know why?
<mars> salgado, that does lazy and/or caching imports, does it not?
<salgado> mars, yes
<mars> salgado, so that would break the import loops, at the expense of making the code look messier :/
<mars> IIRC the function and effect is clear when reading the mainline Zope code
<mars> But I also remember that the I saw were primarily for the purpose of marking functions as deprecated
<salgado> that's one of the functionalities it provides, but certainly not the primary one
<mars> I must have not realized that when I read the code
<flacoste> salgado: i don't, i think gary is the one that made the explanation, but he's offline until Monday
<salgado> mars, the docs ( http://pypi.python.org/pypi/zope.deferredimport) have just a brief mention of how to use it for deprecating things
<mars> losa ping
<mars> oops, wrong channel
<bdmurray> leonardr: is there a special way to remove an exported item in the API or is it as simply as removing exported()?
<leonardr> bdmurray: you need to remove it in a backwards-compatible way (ie. at this point, it has to still be present in beta and 1.0)
<leonardr> is this a field or a named operation?
<bdmurray> leonardr: its a field - can_expire from the review yesterday
<leonardr> i believe i mentioned the syntax in my review. let me check
<leonardr>     can_expire_beta = exported(Bool(...), exported_as='can_expire',
<leonardr>                                ('devel', dict(exported=False))
<bdmurray> yes, I think so
<leonardr> that means it's present in beta and 1.0 but not in devel
<leonardr> or subsequently
<leonardr> what do you think of flacoste's comments?
<bdmurray> I agree with them and will remove unexport can_expire
<leonardr> ok, just remove it totally if you like
<leonardr> by removing exported()
<bdmurray> so it doesn't need to backwards compatible then?
<flacoste> bdmurray: i left this to your judgement in a way :-)
<flacoste> i doubt it was used
<flacoste> but leaving it for backward compatibility isn't that bad either
<flacoste> apart for the cruft it leaves in the code
<bdmurray> well, it was and that is what made us realize people were misinterpreting its meaning
<flacoste> then let's leave it in
<flacoste> for backward compatibility
<flacoste> and to some degree, we need to stick with our policy
<flacoste> otherwise, we'll get burned at some point
<bdmurray> but if they thought it meant something it doesn't mean aren't we helping them?
<bdmurray> and people here is likely 1 or 2 people
<flacoste> I'd say better to get rid of the cruft
<flacoste> then
<flacoste> let's nuke the whole attribute altogether
<flacoste> their script will blow up though
<bdmurray> I was planning to let the consumers I know of know about this
<bdmurray> but really either way is fine with me
<lifeless> where are the current docs for landing an approved branch ?
<james_w> lifeless: https://dev.launchpad.net/LandingChanges  is what I wrote when I researched this for myself. Linked from https://dev.launchpad.net/PatchSubmission, linked from the front page.
<lifeless> thank you
<lifeless> sinzui: ping
<sinzui> hi lifeless
<lifeless> hiya
<lifeless> did you get my mail about profiling?
<sinzui> I did and I think it is fine to keep the config in its module
<lifeless> I'm popping down to reception to pay for the last week we've been here and pickup some soft drink, but if you want to talk about how to make headyway on milestones, I've got time and interest right after that.
<lifeless> sinzui: no, I don't mean the config, I mean for use with analysing the milestones timeouts
<sinzui> Oh, yes I did read your email
<lifeless> those patches were just me cleaning up the stuff I looked at as I pulled on the 'why isn't profiling easy' thread
<lifeless> brb
<thumper> who was chasing the merge conflict between stable and db-devel?
<lifeless> sinzui: so yes
<lifeless> sinzui: you say you haven't made headway; is it just no time, or would you like a hand ?
<sinzui> Not much time this release
<lifeless> no worries
<sinzui> The packaging oopses and feature have precedence.
<lifeless> if you would like a hand, making things fast is a bit of a passion :)
<sinzui> I would like a hand. I am exhausted today, but would like to talk about it latter, may be in a few hours after I eat
<poolie> enjoy your dinner sinzui
<lifeless> I'll be mostly around except for a short gap while I go from hotel to poolie's, who is very graciously letting me use a desk @ his place
<lifeless> anytime you want, ping me
<sinzui> thanks
#launchpad-dev 2010-07-01
<lifeless> hmm, running out of yak shaving time
<lifeless> Anyone care to land an approved branch for me ?
<lifeless> StevenK: ping
<poolie> i got a tcp timeout connecting to bugs.edge...
<lifeless> !
<poolie> just once
<poolie> no other pages failed
<poolie> bug 600000
<_mup_> Bug #600000: missing dependency on Bazaar <hitchhiker (Ubuntu):Fix Released> <hitchhiker (Debian):New> <https://launchpad.net/bugs/600000>
<poolie> big round numbers yay :)
<lifeless> thats really weird as the front end is all load balanced etc
<poolie> exactly
<poolie> lifeless: i realize we can redirect but i wonder if we want to
<poolie> probably
<lifeless> http://www.paulhammond.org/2010/06/trunk/ <- relevant to lp
<lifeless> (mailing the list now)
<lifeless> StevenK: ping
<StevenK> lifeless: pong
<lifeless> StevenK: hi
<lifeless> StevenK: I have two patches I would like to land
<lifeless> but I'm not setup for ec2
<lifeless> StevenK: I'm hioping you can help me
<StevenK> lifeless: Okay, we can either setup ec2, or I can throw them at ec2
<lifeless> please toss em :)
<lifeless> https://code.edge.launchpad.net/~lifeless/launchpad/lsprof/+merge/28583 is one
<lifeless> and https://code.edge.launchpad.net/~lifeless/launchpad/lognamer
<lifeless> sorry
<lifeless> https://code.edge.launchpad.net/~lifeless/launchpad/lognamer/+merge/28574
<lifeless> the lognamer one should go first
<StevenK> lifeless: You should also fix your e-mail address for bzr, it's 'robertc@lpdev'
<lifeless> ugh
<lifeless> danke
<lifeless> I'll add that to the VM set info
<lifeless> fixed for future work
<lifeless> and fixed on the wiki
<adeuring> good morning
<StevenK> lifeless: I finally have your lognamer branch, can you set a commit message in the MP?
<lifeless> done
<lifeless> StevenK: thanks
<lifeless> StevenK: I'll be online later.
<deryck> Morning, all.
<jml> writing about code makes me want to write code instead :(
<lifeless> :)
<lifeless> jml: I've had that intensely this week
<lifeless> jml: though I've also closed off a bunch of bzrloom things, so not as badly as I expect you're going through
<jml> heh.
<jml> my top priority out-of-work hacking task is releasing Twisted
<lifeless> \o/
<jml> which doesn't involve coding
<lifeless> does it have the fixed issue with subunit and fake errors ?
<lifeless> you could release testtools if you want
<lifeless> oh right, coding
<jml> 10.1 has the subunit fix, yes.
<lifeless> what would you like to write code on
<jml> lots of LP stuff; twisted release automation; a couple of testtools fixes I have in mind; some tweaks for my graphing thing
<lifeless> cool
<lifeless> ngight all
<bigjools> jml: don't know if you saw my message yesterday, at least I don't remember seeing a reply.  Are you going to be able to look at those buildd-manager changes?
<jml> not soon, no.
<mwhudson> bigjools: is this twisted-y stuff?
<bigjools> mwhudson: very much so
<mwhudson> bigjools: i might be able to have a look, although my brain is made of cotton wool today
<bigjools> mwhudson: it's best if I take you through it, the buildd-manager is a, err, special piece of code
<bigjools> but it'll have to wait until next week if that's ok?
<mwhudson> bigjools: next week should be fine
<mwhudson> and yes, i've looked at buildd-manager before
 * mwhudson rubs the mental scar tissue
<bigjools> I still have bits of fingernail in my eyes
<bigjools> it was only after I started re-writing bits of code that I realised a lot of the crap in there is explicitly for tests
<mwhudson> bigjools: what are you working on?  just generally improving the sanity level?
<bigjools> mwhudson: when it was developed its design was influenced too much by the old scanner than did one massive scan over the whole farm.  I changed the behaviour so that I've got separate Deferred threads for each builder in the farm, so that when the scan fails it doesn't blow out the scan of the whole farm.
<bigjools> and it's a lot faster generally with that approach as it can do more stuff in parallel
<mwhudson> bigjools: sounds good
<bigjools> however, this has completely broken the existing tests of course :)  I need advice on how to write some sane tests.
<mwhudson> although 'developed' is perhaps an insufficiently perjorative term? :)
<bigjools> you are indeed the master of understatement
<bigjools> mwhudson: if you want to take an advanced look, the branch is here:   https://code.edge.launchpad.net/~julian-edwards/launchpad/buildd-manager-parallel-scan
<bigjools> now, I need fud
<jtv> danilos: you may still be too busy for this, but I made a few tweaks to the decision matrix we worked out for setCurrentTranslation.
<danilos> jtv, it's wrong, totally wroooong!
<jtv> uh-oh
<danilos> jtv, I'll take a look
<jtv> and _then_ tell me it's all wroooong.  :-)
<danilos> jtv, btw, I should look at the wiki page or code?
<jtv> danilos: wiki page would be easier I think, though the code should match it pretty closely
<leonardr> salgado-lunch or flacoste_afk, i have a launchpad/web service problem that you might be able to help with when you return
<salgado> hi leonardr
<james_w> hmm, got an ec2 run that is failing every test
<james_w> "Warning: 'with' will become a reserved keyword in Python 2.6"
<james_w> and then failing with syntax error when it tries to use "with"
<leonardr> salgado: this problem has to do with navigation
<james_w> running with 2.4 python?
<jml> james_w, is the test failing locally?
<leonardr> i am publishing ~{user}/oauth_access_tokens as a collection of IOauthAccessToken objects
<james_w> jml: no
<james_w> jml: *every* test is failing in ec2
<leonardr> the problem is that IOAuthAccessToken objects are not currently published on the website. so they have no canonical_url and there is no way to navigate to them
<jml> james_w, oh.
<leonardr> i hacked in a canonical_url but i can't figure out how to make navigation work
<leonardr> is this something you can help with?
<jml> james_w, I saw a patch for something similar sounding fly by today
<james_w> jml: ah yeah, thanks, I'll try merging that
<leonardr> salgado: once i get the navigation working there is another problem with permissions and oauth access levels but we'll deal with that later
<salgado> leonardr, IIUC, what you want is a traversal so that you can get to the OAuth token?
<leonardr> salgado: right
<leonardr> i was trying to make /~leonardr/oauth_access_tokens/{key} work, but that wasn't working
<leonardr> i looked at other deeply-nested objects like bug messages, and i didn't see any way of recreating their complex navigation code for something that was 1. part of the main application rather than malone, and 2. published only in the web service
<salgado> leonardr, so, something similar to what we do for email addresses?
<salgado>     @stepthrough('+email')
<salgado>     def traverse_email(self, email):
<salgado>         """Traverse to this person's emails on the webservice layer."""
<salgado>         email = getUtility(IEmailAddressSet).getByEmail(email)
<salgado>         if email is None or email.personID != self.context.id:
<salgado>             return None
<salgado>         return email
<leonardr> that is promising! where's that code?
<salgado> lib/lp/registry/browser/person.py
<jml> leonardr, salgado: not necessary, but it would be nice to have a short howto / checklist put up somewhere as a result of this conversation.
<leonardr> jml: yeah, i know it's come up before
<salgado> indeed
<jml> glad you agree :)
<salgado> leonardr, note you'll have to use a different path segment than the one you use for the collection (e.g. +oauth-access-tokens)
<leonardr> right
<leonardr> should it be singular or plural? i see singulars here
<salgado> singular
<james_w> https://dev.launchpad.net/Web/URLTraversal
<jml> oh look there's a thing :)
<jml> james_w, thanks.
<james_w> np
<salgado> leonardr, you might want to give that wiki page a try and update it in case you find anything else
<leonardr> salgado: "Traverse to this person's emails on the webservice layer." is that "webservice layer" part enforced at all?
<salgado> leonardr, I don't think so
<leonardr> ok
<salgado> it's not indeed -- one can traverse to an email address on the web app
<sinzui> oh sweet
<sinzui> I can do the same searching /people/ by email address, but that is kind of secret and definitely slower than API
<james_w> jml: may I move FileIsADirectory from lp.codehosting.sftp to lp.services.sshserver?
<james_w> jml: or a better place?
<leonardr> salgado: now that we've resolved that, i'd like your advice on implementing a new OAuth access level and the permissions that go with it
<leonardr> http://pastebin.ubuntu.com/457940/
<leonardr> i can create the access level itself easily enough (#1) but i'm not sure how the very specific permission control we need for it can work given our existing permission system
<salgado> leonardr, I think you'll need to special case IOAuth* objects in LaunchpadSecurityPolicy.checkPermission() because the security adapters don't have access to the oauth token used to authenticate, so they can't see if the access level is GRANT_PERMISSIONS or not
<salgado> leonardr, but you should talk to gary/flacoste before going forward with that, because it's a bit of a hack
<leonardr> salgado: agreed. this design is one i came up with with gary, but i don't know if he knows about that implication
<salgado> leonardr, another (even more hacky, although less intrusive) option would be to use get_current_browser_request() in your security adapter to have access to the oauth token
<leonardr> salgado: security adapter for IOAuthAccessToken? can you point me to a similar example?
<salgado> leonardr, lib/canonical/launchpad/security.py
<salgado> leonardr, these are called by LaunchpadSecurityPolicy.checkPermission(), but they're only given the logged in user and the object to check permission on
<leonardr> i think that might be a better solution
<salgado> leonardr, btw, do you have a few minutes to help me with lazr.restful?
<leonardr> salgado, sure
<salgado> leonardr, AIUI, the way lazr.restful exports things is by annotating the interfaces and their methods/attributes and later scanning them to generate the wadl to describe the service. is that correct?
<leonardr> salgado: yes
<salgado> leonardr, and how does the web app maps the requests back to say, method calls, later?
<leonardr> salgado: the scanning process actually generates a set of adapter classes (adapting the various data model classes to IEntry)
<leonardr> those adapter classes are used to generate the wadl and also to turn http requests into operations on the data model objects
<salgado> oh, right, I'd forgotten about the adapters
<salgado> leonardr, cool, I think I know enough to move forward now.  I'm doing a proof of concept for exporting things that are not directly provided by a given class but can be provided through an adapter (like what james_w described in that Adapters & API thread
<thumper> morning
<lifeless> hiya
#launchpad-dev 2010-07-02
<lifeless> sinzui: ping
<lifeless> I'm looking at https://edge.launchpad.net/ubuntu/maverick/+source/upower/+edit-packaging
<lifeless> and and https://edge.launchpad.net/upower which seems to be totally different
<lifeless> whats the process to zap /upower so someone can link the udev upower to it ?
<wgrant> lifeless: /upower seems to be deactivated -- I can't see it.
<lifeless> wgrant: it is
<lifeless>  its for a 'User POWER on' crack feature
<lifeless> mars: have you seen 'shrinksafe' ?
<poolie> i wonder why my dapper ppa only seems to be building on i386
<lifeless> because its arch:all
<poolie> ah of course
<nigelb> bryceh: Happy Birthday! Have a splendid time!
<adeuring> good morning
<mrevell> Howdty
<wgrant> bigjools: Morning.
<bigjools> g'day
<bryceh> nigelb, thanks :-)
<wgrant> bigjools: This granting of launchpad.View to P3A subscribers does not sit well with me. I've not seen any rationale for changing the very explicit decision to not grant it in the first place, and it introduces unobvious things like bug #600910.
<bigjools> le sigh
<bigjools> wgrant: it's the right thing to do generally, however yes there's going to be some wrinkles
<nigelb> bryceh: :)
<wgrant> bigjools: Well, yes, but it would seem wise to identify and iron out the wrinkles before granting a whole lot of unintended privileges...
<bigjools> wgrant: we thought we had....
<bigjools> at least the ones we saw were not important right now
<spm> bigjools: that comment was below the belt (ref wallabies). That is all. ;-)
<bigjools> spm: bwahaha :)
<wgrant> bigjools: So the +packages restriction is just to... avoid confusing users? Because all the other subordinate pages and API stuff is still permitted.
<bigjools> spm: the series in Sydney just gone was very entertaining though
<bigjools> wgrant: yes that's still a wrinkle that needs to be ironed, but we don't care about it right now
<spm> heh, I'll have to take your word for it. wasn't even aware it was in Sydney.
<wgrant> OK.
<bigjools> wgrant: basically because it only really matters when we do commercial PPAs
<bigjools> wgrant: anyway thanks for filing the bug
 * bigjools proceeds to download another load of security fixes. sigh.
<wgrant> Heh.
<vila> hi guys,
<vila> regarding bug #525571, what bzr and bzr-svn versions should I target to simplify the fix backport on launchpad ?
<_mup_> Bug #525571: No locking when updating files in ~/.bazaar <canonical-losa-lp> <code-import> <udd> <Bazaar:In Progress by vila> <Bazaar Subversion Plugin:Triaged> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/525571>
<vila> wow, _mup_, you're far verbose than the ubot<n> :)
<vila> far more verbose
<lifeless> it is entertaining when they were in the same channel.
<vila> http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/annotate/head:/versions.cfg says 2.1.0 for bzr but doesn't mention bzr-svn
<rinze> vila: 'morning
<vila> omg, yet another nick :)
<lifeless> jelmer!
<vila> jelmer is around ? Cool :)
<rinze> hi lifeless
<rinze> vila: bzr-svn trunk should already support LockableConfig
<vila> rinze: wow, supporting un-landed fixes is an impressive break-through :)
<vila> rinze: I guess that answer my question by implying that bzr-svn trunk will be back ported right ?
<vila> errr, deployed I meant
<rinze> vila: Yeah, the code team usually just take a known working version of bzr-svn
<rinze> s/version/snapshot/
<vila> ok, I'll focus and targeting the fix to bzr-2.10 then
<vila> thanks
<vila> <shudder> another typo day...
<lifeless_> _fail_ networking.
<lifeless_> this E160 is so bad
<deryck> Morning, all.
<lifeless> morning
<lifeless> so you were asking what I mean by 'trunk can be broken'
<lifeless> I mean a few things, most of them summarise as 'not fit for purpose', I think.
<lifeless> most obviously, we currently don't trust trunk
<lifeless> when we release, we insist on a QA step done by each team, for many changes
<lifeless> also, the way we do commits to trunk, we can have tests failing
<bigjools> edge trunk = stable though
<lifeless> could you expand on that, I'm not entirely sure I'm across everything here.
<bigjools> we commit to "devel" which is blessed by buildbot and merged to stable
<bigjools> edge rolls out from stable
<bigjools> so it's slightly disingenuous to talk about trunk
<lifeless_> 20:27 < lifeless> could you expand on that, I'm not entirely sure I'm across everything here.
<lifeless_> 20:27 -!- zyga_away is now known as zyga
<bigjools> gar
<lifeless_> and then I lost the magic network
<bigjools> <bigjools> we commit to "devel" which is blessed by buildbot and merged to stable
<bigjools> <bigjools> edge rolls out from stable
<bigjools> <bigjools> so it's slightly disingenuous to talk about trunk
<lifeless_> ah
<bigjools> we have 4 trunk branches
<bigjools> all slightly different :)
<lifeless_> what does the merge to stable entail - full test suite ?
<mwhudson_> dev.launchpad.net/Trunk and dev.launchpad.net/Trunk/Glue ftw
<lifeless_> you are in a mazy of twisty tunnels.
<lifeless_> There is a grue.
<lifeless> bigjools: what does the merge to stable entail - full test suite?
<wgrant> devel -> full test run -> stable
<wgrant> stable -> full test run -> db-devel
<wgrant> Er.
<wgrant> No test run in that second one, sorry.
<wgrant> db-devel -> full test run -> db-stable
<zyga> lifeless, ?
<lifeless_> mwhudson_: thanks
<lifeless_> your links were the last things I saw.
 * lifeless_ hates on this gsm modem.
<bigjools> lifeless: I'm trying to find a wiki doc explaining this, but basically wot wgrabt says devel -> buildbot -> merge to stable -> merge to db-devel -> buildbot -> merge to db-stable
<zyga> lifeless_: 20:27 -!- zyga_away is now known as zyga <- ?
<lifeless_> zyga: showing what I saw before d/c
<zyga> lifeless_, ah ok
<lifeless_> so *stable has no test failures modulo really unexpected things
<wgrant> lifeless_: Right. Any issues in *stable are holes in the test suite.
<lifeless_> <-hate->
<deryck> Sorry I missed the discussion here.  I thought lifeless meant take it to the launchpad-dev mailing list.
<lifeless> deryck: no worries,w list works too :)
<lifeless> deryck: I wouldn't normally take a realtime conversation to the list though, for future ref
<deryck> lifeless, understood.
<deryck> thanks for the discussion, though.  I really hope we can end up with some form of this.
<lifeless> Me too
<lifeless> I'm trying to make sure we exam it critically
<lifeless> so that its a success
<deryck> yes, I don't mind critical examination at all.  I want it's success, too.
<lifeless> :)
<deryck> "its" even.  Perhaps I stayed up too late playing Champions Online last night.
<lifeless> StevenK: ping
<lifeless> StevenK: what happened with that lognamer branch ?
<lifeless> deryck: perhaps we can drill into this at the epic
<deryck> lifeless, yeah, I think we should somehow.  Not sure how to drill into it in a productive way, though.  talk or code?
<lifeless> quick gap analysis, identify the most beneficial thing in the gap to fix, fix it.
<lifeless> hey, so how do I land an approved branch if I'm not ex2test-ready
<lifeless> https://code.edge.launchpad.net/~lifeless/launchpad/lognamer/+merge/28574
<deryck> lifeless, I can `ec2 land` it for you.
<lifeless> awesome
<lifeless> if it fails, do I get mail ?
<deryck> lifeless, and that plan sounds reasonable.  An analysis talk, followed by fix something, followed by list of remaining items to fix.
<deryck> lifeless, yes, you'll get mail.
<lifeless> \o/
<lifeless> deryck: in general I think knowing the scope of the problem is useful, but detailed talking beyond the first couple of steps is generally wasted, unless you have reason to believe a roadblock will be hit
<deryck> lifeless, right, agreed.  I think we need a way to better connect lp devs and losas on this issue, too.
<lifeless> s/on this issue, /
<lifeless> :P
<lifeless> theres a pretty good collaboration about operational issues, but significant disconnects in other ways
<deryck> right :-)
<wgrant> Wow. i386 build queue up to 24 hours.
<lifeless_> jml: https://edge.launchpad.net/testtools/0.9/0.9.4 done
<StevenK> lifeless_: I did send it off to ec2, so that's just odd
<lifeless_> StevenK: yeah
 * rinze waves to StevenK
<lifeless_> StevenK: I trust that you did
<lifeless_> deryck has done it again
<lifeless_> StevenK: I didn't get mail
<StevenK> Ah, okay
<lifeless_> rinze: your older nicks are better
<jml> lifeless_, thanks!
<lifeless_> jml: my pleasure
<deryck> what did I do?
<lifeless_> deryck: ec2test my branch
<deryck> ah, ok
<lifeless_> I really want an 'unreleased code report' for my projects.
<lifeless_> perhaps next year I'll shave that yak.
<lifeless_> I really wish releases were announcements
<lifeless_> http://pypi.python.org/pypi/celery/2.0.0 looks interesting
<lifeless_> and with that, I think I'm going to halt()
<lifeless_> hmm
<lifeless_> I smell inappropriate coupling
<lifeless_>  "/var/launchpad/test/lib/lp/services/job/tests/test_runner.py", line 410, in runFromSource
<lifeless_>    errorlog.globalErrorUtility.oops_prefix,
<lifeless_> for the weekend
<lifeless_> deryck: thanks for that.
<bigjools> lifeless_: celery is something I'm interested in
<lifeless_> bigjools: @ the epic, lets talk
<bigjools> yeppers
<bigjools> lifeless_: I'm thinking build farm
<lifeless_> bigjools: we have many things that look like an appropriate nail, to me.
<bigjools> hellyeah
<bigjools> we *need* a messaging system now
<bigjools> hanging a job system off it would be great
<wgrant> Is there a way I can get an up-to-date milestone bug listing?
<wgrant> I'm trying to garden an upcoming milestone, but working blind is a bit hard.
<deryck> wgrant, you mean un-cached?  If so, I don't think there is a way.
<wgrant> Argh.
<wgrant> That sucks.
<wgrant> Ah, production is untainted for the next few days.
<wgrant> This is good.
<wgrant> Bug #601051
<_mup_> Bug #601051: Caching of primary data sources slashes utility <Launchpad Registry:New> <https://launchpad.net/bugs/601051>
<wgrant> bigjools: How is the rebuild archive scored at the moment? All -10, except for main, which is -5?
<bigjools> wgrant: not quite
<bigjools> stuff is getting scored up in batches to make it run quicker
<wgrant> Ah.
<wgrant> I was wondering why there was so much stuff building.
<wgrant> When the massive PPA queue should have been building instead...
<bigjools> there's not much of it left, it'll get disabled when main is done
<wgrant> Great.
<wgrant> Because it's... erm... somewhat irksome when using PPAs to manage release distribution, and the queue is a couple of days long.
<maxb> How come there's never a help contact in the #launchpad topic these days?
<jtv> Anyone know why my Librarian won't restart after an unexpected system shutdown, and what I do about it?
<mwhudson> jtv: stale pid file?
<mwhudson> in /var/tmp/ somewhere i guess
<EdwinGrubbs> jelmer_: ping
<jelmer_> EdwinGrubbs, pong
<jelmer_> EdwinGrubbs, pong
#launchpad-dev 2010-07-03
<lifeless> what product should I file a bug on to file a bug about the source address of ec2test failure mails.
<wgrant> lifeless: launchpad-foundations.
<lifeless> anyone with landing access around ?
<lifeless> anyone with landing access around ?
<lifeless> anyone with landing access around ?
<lifeless> hmm, I really need to get ec2test enabled.
<mwhudson> lifeless: what do you mean?
<lifeless> mwhudson: I have branches to land, but haven't gotten around to figuring out how to use ec2test
<lifeless> Added to which my ec2 credentials are outside my dev vm, I don't really want credentials *in* the vm, so I need to look at whether there is an agent facility too
<lifeless> mwhudson: asking folk to land stuff for me isn't a good long term strat.
<mwhudson> you have seen https://dev.launchpad.net/EC2Test ?
<mwhudson> i don't think there's anything like an agent for ec2 creds
<lifeless> yes
<lifeless> so I need to copy the ec2test utility out, and whatever deps it has
<lifeless> debug
<lifeless> etc
<mwhudson> you could probably just run it from a launchpad tree, it *might* not require the deps to be present
<mwhudson> although python-boto versions will probably bite you
<lifeless> mwhudson: I don't have any lp trees outside the vm :)
#launchpad-dev 2010-07-04
<mwhudson> well, if you /want/ to make life hard for yourself...
<lifeless> not really
<lifeless> I'm a bit short on disk right now, which is adding to the mess
<lifeless> which reminds me, I think aptitude is failing to give me back a gb or three
<lifeless> ah yes, 1.2GB back
<lifeless> thank you linux-image
<lifeless> mwhudson: may entertain you - I just found a knit1 version of lp on my disk
<mwhudson> getting rid of that should save some space :)
<mwhudson> unless it's terribly old, i guess
 * mwhudson zzzz
<lifeless> gnight
<lifeless> righto, bombs away on ec2test. Time to sleep
<mwhudson> gosh make build takes a long time
<jml> mwhudson, 'make compile' is almost always enough.
<mwhudson> ah
 * mwhudson hmms
<mwhudson>     ImportError: No module named uuid
<mwhudson> oh right, sourcecode
<adiroiban> n
<lifeless> doesn't it piss you off when you can't find a web site you knew a year ago
#launchpad-dev 2011-06-27
<lifeless> dynamic port allocation \o/
<nigelb> lifeless: You aren't in Dublin?
<lifeless> nope
<nigelb> Ah.
<lifeless> grah
<lifeless> and it has no setup.py etc
<lifeless> I think I'll extract code from LP
<nigelb> lifeless: any specific reason you picked web.py?
<lifeless> it wasn't zope, it wasn't django, and it is in lucid
<nigelb> heh
<nigelb> I meant among the other microframeworks
<lifeless> so do I
<nigelb> ah, in lucid.
<lifeless> another option would be pastedeploy
<lifeless> but its possibly so minimal as to not be a framework anymore
<nigelb> Never heard of it. ::looks::
<bac> lifeless: ping
<lifeless> hi bac
<bac> lifeless: abentley is trying to skype you so you can participate
<bac> ah, nm, i hear martin connected
<lifeless> I will irc if I have stuff to interject
<lifeless> iron stomach, no seasickness
<lifeless> someone tell them not to fuss to much :)
<lifeless> I see a long corridor between people at tables
<jcsackett> lifeless: they seem to be enjoying it; we would hate to stop their fun.
<StevenK> It's a room!
<lifeless> hi
<lifeless> yes I hear you
<lifeless> 'I can see you alll... muahhahahahahaha'
<vila> :)
<jml> lifeless: watching here in case you want to interject. use my nick.
<lifeless> jml: thanks
<lifeless> jml: looking at the room is cute; looking at the slides might be better ?
<jml> lifeless: ok.
<lifeless> jml: thanks
<jml> lifeless: np. is that an ok view? (I'm not very close to your laptop)
<lifeless> its fine
<jml> cool.
<lifeless> jml: at a convenient point please note 'if we are willing to spend more time on db patches, we have stakeholder goahead to make db patches go live 24 hours after landing'
<lifeless> jml: we now have a lot of infrastructure/db work to do to to solve timeouts
<lifeless> http://boom.cs.berkeley.edu/ is interesting (and totally unrelated to the talk going on :))
<lifeless> jml: whats thing flacoste is saying - I can't make out the noun
<jml> lifeless: talking about using cloud stuff, incl ensemble, for developing Launchpad
<lifeless> ah
<wgrant> lifeless: Any reason we shouldn't deploy today?
<jml> one thing, not worthy of interjecting, is that anything that makes it harder for new contributors also probably makes it harder for established developers on a day-to-day basis
<jml> e.g. imagine if we could spin up a build farm with ten machines locally using LXC
<jml> with one command
<lifeless> I think for adhoc stuff like that, ensemble may be a wonderful fit
<jml> yeah, me too
<StevenK> wgrant: Fixed.
<wgrant> StevenK: Thankyou sir.
<wgrant> Now I need to find wallyworld.
<StevenK> He's behind you!
<lifeless> jml: for the unit test suite, things like lxc (alone, nevermind layers on top of) seem to be much slower than I would like to aim at
<jml> lifeless: interesting.
<jml> lifeless: do you want to watch benji's talk?
<lifeless> jml: coffeescript?
<lifeless> no; its pretty cool stuff though.
<lifeless> but I have stuff I need to go do.
<jml> lifeless: ok. thanks.
<lifeless> have a great day guys; I'm signing off.
<lifeless> bigjools: btw, I think bug 802335 is a side effect of the new packagecopyjob table
<_mup_> Bug #802335: DistroSeries:+queue timeouts when filtering by package name <regression> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/802335 >
<lifeless> bigjools: I can't talk now, but perhaps we can make 5 minutes tomorrow - *if* it is the new table, I raised the impact it would have on +queue during review - I'd like to drill into how we let this through
<lifeless> bigjools: (I'm mentioning it now so you aren't surprised when I try to grab you tomorrow to talk about it :))
<bigjools> lifeless: jtv changed the query significantly so grab him :)
<bigjools> but happy to talk if you want
<Ursinha> danilos: salinas, right?
<NCommander> StevenK: pinbg where are you?
<jml> lifeless: if you're around... where are the architecture values that were in your talk?
<jml> lifeless: never mind.
<jml> <https://dev.launchpad.net/ArchitectureGuide/ServicesRequirements> is what I was actually looking for
<jml> since any long poll stuff we do has to comply with all of that.
<StevenK> NCommander: I am on level 1. What's up?
<poolie> spiv, danilos, jelmer, jtv, https://dev.launchpad.net/Projects/LiveBranches#preview
<NCommander> StevenK: infinity found my issue (LP gave a non-helpful error message on a reject)
<StevenK> NCommander: How unhelpful?
<NCommander> StevenK: I did an upload to oneiric which was rejected due to a -20 existing in proposed, it complained my tarball was wrong (checksum mismatch)
<StevenK> NCommander: Right, that isn't unhelpful. :-P
<LPCIBot> Project devel build #840: FAILURE in 23 min: https://lpci.wedontsleep.org/job/devel/840/
<StevenK> Hmmm
 * StevenK pokes at Jenkins
<StevenK> Ah ha. Gary broke it.
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #671: FIXED in 5 hr 51 min: https://lpci.wedontsleep.org/job/db-devel/671/
<jml> gmb, wgrant: http://paste.ubuntu.com/633629/
<jml> bigjools: ^^
<jml> storm, transaction, twisted
<jml> txamqp
<jml> zope.component, zope.schema, zope.interface, zope.configuration
<jml> huwshimi: are you near deryck?
<huwshimi> jml: I think he's about 5 metres away
<huwshimi> jml: Certainly within throwing distance
<jml> huwshimi: what about gary?
<huwshimi> jml: An equal distance away
<wgrant> Neither is on IRC :(
<jml> huwshimi: excellent. the build is broken, and gary broke it
<wgrant> But gary broke devel, somehow.. and it corrupts
<StevenK> I have landed a revert
<wgrant> corrupts the setuptools egg.
<wgrant> buildbot is probably still broken.
<wgrant> Because its egg is broken.
<huwshimi> jml: Do you need me to throw something?
<StevenK> jml: I am sitting next to deryck
<jml> StevenK: please ask gary to fix it :)
<StevenK> jml: Both flacoste and gary are working on it right now.
<jml> sweet.
<jml> StevenK: thanks.
<flacoste> jml: we need to remove eggs/setuptools*
<flacoste> because distribute is evil
<flacoste> really evil
<flacoste> most evil force in the universe
<Ursinha> haha
<Ursinha> matsubara: do you know if soybean oopses are synced to devpad every 10 minutes or once a day?
<matsubara> Ursinha, I think it's every hour now for all oopses
<Ursinha> oh
<Ursinha> that's bad
<jml> gmb: dependencies: storm, transaction, twisted, txamqp, zope.component, zope.schema, zope.interface, zope.configuration
<jml> stub: http://as.ynchrono.us/2007/12/filesystem-structure-of-python-project_21.html (fwiw)
<jml> bigjools: http://twistedmatrix.com/documents/current/core/howto/tap.html
<allenap> wgrant, jml, stub: http://paste.ubuntu.com/633675/
<wgrant> allenap: You are a horrible person.
<allenap> wgrant: Thank you :)
<StevenK> Haha
<StevenK> abentley: O hai.
<abentley> StevenK: Hai.
<StevenK> abentley: Do you remember what the JS that is loaded after page load on projectseries/+setbranch does?
<nigelb> Is it known that on the branch page, if I click diff, I see the diff. After that the links are no longer "clickable"
<abentley> StevenK: Nothing is coming to mind.
<jml> wgrant: http://www.cs.yale.edu/quotes.html
<jml> bigjools: http://twistedmatrix.com/documents/current/core/howto/application.html
<huwshimi> jcsackett: http://gfxmonk.net/shellshape/
<nhandler> Is https://help.launchpad.net/API/SigningRequests outdated? I keep getting an error saying oath is an unsupported authentication method when I attempt to post to /+request-token
<wgrant> huwshimi: Huh, I know him.
<wgrant> From uni.
<huwshimi> wgrant: Oh right
<wgrant> Oddity.
<poolie> nhandler: nb it's oauth not oath
<poolie> they are both authentication mechanisms
<nhandler> Bleh. Thanks poolie. I really shouldn't code before fully awake :)
<poolie> not the best naming system in the world
<jcsackett> bac: do you know if bug 705153 is still valid? it looks to me like the hover over data on subscribers has been removed outright, which might make this [wontfix|fixreleased].
<_mup_> Bug #705153: "Subscribed by" hover is incorrect <Launchpad itself:Triaged> < https://launchpad.net/bugs/705153 >
<bac> jcsackett: you are correct.  i've marked it invalid
<spiv> wgrant: here's some  loggerhead code you may like:
<spiv> def dq(p):
<spiv>     return urllib.quote(urllib.quote(p, safe=''))
<wgrant> spiv: Burn
 * maxb chuckles
<nhandler> Any ideas why I get a 'Missing Authenticate Header' client-warning in my script (OOPS-2004STAGING186) ? I have an Authroization header that follows the documentation (I already followed the guide to get a request token and then an access token)
<Auree> Hi, I couldn't find it in the documentation, what is the limit for client applications to the Launchpad API?
<LPCIBot> Project parallel-test build #75: STILL FAILING in 1 hr 19 min: https://lpci.wedontsleep.org/job/parallel-test/75/
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 204 - 0:[######=_]:256
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #841: FIXED in 5 hr 49 min: https://lpci.wedontsleep.org/job/devel/841/
<lifeless> moin
<stub> yo
<stub> think most people are at dinner
<lifeless> cool
<lifeless> how was the day?
<stub> Seemed to go ok. Getting some traction on rabbit.
<stub> Catching up from last week too.
<stub> Not sure how the other groups fared
<lifeless> I haven't filed bugs for the fast-downtime stuff yet
<lifeless> james_w: speaking of which; ping
<lifeless> stub: I will file a few today and assign to you so you can see them; they don't have to be worked on this week - do stuff with the folk there; (but if you do work on it its pretty exciting stuff :P).
<timrc> wgrant, lifeless, jcsackett: thank you guys for such a quick turnaround time on our api enablement requests, greatly appreciated
<james_w> lifeless, hi
<lifeless> james_w: I don't recall hearing back from linaro on stakeholders about db schema downtime
<james_w> lifeless, I haven't heard anything, I'll prod, but otherwise we can take it as "silence is agreement" and move ahead
<lifeless> thanks
<wgrant> lifeless: You may have seen the rabbit layer land.
<wgrant> lifeless: We sorted out the failures.
<lifeless> great
<lifeless> what were they ?
<wgrant> And ripped canonical.amqp out of landscape, into lazr.amqp
<wgrant> Well.
<wgrant> The problem was the config overlay is just about impossible to remove.
<wgrant> Since it's written to disk.
<wgrant> So we just made the layer unteardownable.
<lifeless> that reminds me, I needed to talk with flacoste about lazr :P
<wgrant> We didn't want to call it lazr, but it was easiest for now.
<lifeless> wgrant: what did you want to call it ?
<wgrant> No ide.
<wgrant> idea
<wgrant> Which is why it's lazr.
<wgrant> It also currently uses the system rabbit for tests :(
<lifeless> wgrant: unteardown makes me sad; we use config overlays on disk for librarian too, why is that different ?
<wgrant> But I've ported it mostly to the fixture.
<lifeless> and db
<wgrant> lifeless: I don't quite recall.
<wgrant> lifeless: But the tests want the config overlay to be absent.
<lifeless> could someone file a bug? low tech-debt
<wgrant> I suspect the librarian/db tests do not.
<wgrant> eg. the librarian hiding thing just sets a bogus port.
<wgrant> Also, how do I make efficient use of a fixture?
<wgrant> ie. use .reset() between tests?
<wgrant> Rather than restarting rabbit each time.
<wgrant> Which is slow.
<wgrant> (this is in lazr.amqp, so no layer)
<jml> also
<jml> stub was saying that zope now has a way to tear down the functional layer
<wgrant> Yeah.
<wgrant> We should be able to.
<wgrant> But lazr.config needs some wrangling before we can do it for everything.
<jml> if we get rid of that, the new unteardownable thing and wgrant gets his question answered, we could stop using layers
<wgrant> Yay
<wgrant> BURN
<wgrant> etc.
<jml> that's three things that are non-trivial before we get to do the thing we actually want to do, which is also non-trivial :)
<lifeless> jml: there is a zope.testing.functional or something which now has a tearDown
<lifeless> we need to test using it
<lifeless> I don't knove if the CA has a teardown yet
<lifeless> wgrant: .reset() exists for use between tests, yes.
<jml> you could probably hack one up
<wgrant> lifeless: But how do I use it?
<jml> replace globalregistry with a new one
<jml> wgrant: aye, and therein lies the rub
<lifeless> wgrant: you use the unwritten testfixtures optimiser code
<lifeless> wgrant: -or-
<lifeless> wgrant: and I recommend this, use testresources and write a -very- shallow thunk from a ResourceManager to the fixture.
<wgrant> Hmm. Perhaps.
<wgrant> Thanks.
<jml> if it's run with zope testrunner, that won't help though
<wgrant> It is the zope testrunner at present :(
<lifeless> if its run with zope testrunner you can use a layer
<wgrant> True.
<wgrant> But it also workswith trial, which I don't reallly want to break.
<lifeless> when you said foo so no layer, I assumed a good testrunner
<lifeless> wgrant: choose.
<wgrant> So there's no cross-runner compatible way to do it?
<wgrant> Sad.
<lifeless> wgrant: there is, if you fix zope testrunner to not destroy test suites
<lifeless> e.g. make it compatible
<wgrant> lolz
<lifeless> FWIW, I'd do the following:
<lifeless>  - make testr the primary runner for it (implies trial compat)
<lifeless>  - use testresources
<lifeless>  - profit
<wgrant> Possibly.
<lifeless> you can use trial --reporter=subunit
<wgrant> We're using zope.testrunner now because it was easy with buildout, but I gues we can just use testr with something.
<wgrant> Aha.
<wgrant> Handy.
<lifeless> it doesn't have load-by-id for broken-test-iterations or parallel support
<wgrant> :(
<lifeless> but if you use jmls' most excellent twisted support in *testtools*
<wgrant> Asny suggestions for something better?
<wgrant> Ah.
<lifeless> then you're sitting pretty
<wgrant> I looked at that, but it said it was highly experimental and not to be used.
<lifeless> lies
<lifeless> we use it in Launchpad
<lifeless> its more 'subject to change'
<wgrant> Ah.
<lifeless> wgrant: jml: I'm glad you guys are hacking on this.
<wgrant> "testtools provides highly experimental support for running Twisted tests â tests that return a Deferred and rely on the Twisted reactor. You should not use this feature right now. We reserve the right to change the API and behaviour without telling you first."
<wgrant> But I guess if you and jml tell me otherwise...
<lifeless> wgrant: so what does lazr.amqp do?
<jml> wgrant: we use it consistently in Launchpad
<lifeless> wgrant: is it landscapes long poll service?
<lifeless> wgrant: or just glue code?
<wgrant> lifeless: Mostly a Twisted HTTP server for rabbit-based long-polling, yes.
<wgrant> It also has a rabbit job runner.
<lifeless> so, I think the name is totally misleading
<lifeless> lazr.longpoll perhaps
<wgrant> Possibly.
<jml> it's actually more "I don't want people to complain too much if it's broken"
<wgrant> The name is not really defined.
<jml> txlongpoll?
<wgrant> It was canonical.amqp, and Julian JFDI.
<lifeless> jml: +1
<lifeless> jml: that would be so much better
<jml> lifeless: our main aim is to get something working end-to-end
<lifeless> sure, cool
<wgrant> It's not *just* long polling, as it turns out.
<lifeless> so the reason I asked
<wgrant> But that's one of its modules.
<jml> lifeless: I'm very much in favour of a better name, but will only change it if I'm confident that we'll have something demonstrably functional before EOW
<jml> (of course anyone else can change it)
<lifeless> so, you've seen ServicesRequirements
<jml> fwiw, I am pretty confident :)
<lifeless> I want to highlight a couple of the implications there
<jml> lifeless: yes, https://dev.launchpad.net/Projects/LongPollNotes links to it
<lifeless> we shouldn't be importing *any* of the code in this project into LP itself.
<jml> lifeless: how is that implied?
<lifeless> https://dev.launchpad.net/ArchitectureGuide/ServicesRequirements#Test fake
<lifeless> its not mandatory [yet]
<lifeless> it will be soon
<jml> ahh good. NTH.
<lifeless> NTH ?
<jml> and also something that can be fixed late
<lifeless> so when I say test fake
<jml> lifeless: nice-to-have. sorry, it turns up a lot in my notes for things.
<lifeless> I mean 'live network service that is -fast- to startup, uses dynamic allocated ports and supports error injection
<lifeless> I specifically do not want in-process fakes
<lifeless> because they increase coupling and tighten version dependencies
<jml> lifeless: why do you object to the package having its own client code & test double?
<lifeless> I want to thoroughly decouple things
<lifeless> we're going to be deploying entirely separately
<lifeless> if its physically impossible to do a lockstep change to the client and server
<lifeless> I think it will really help people avoid such disasterous changes
<lifeless> it also works better when we rewrite a service to be in a faster language
<lifeless> and avoids fugly around needing N clients (one for tx, one for python, one for javascript) in the same tree - or alternatively inconsistently blessing one as 'this one is in-tree, the others are not'
<lifeless> thats the angle for client code
<lifeless> for test double, I *do* want the service to provide its own test double.
<mwhudson> lifeless, jml: did you know that the django test runner manages to be EVEN MORE terrible than the zope one?
<lifeless> I just want that to be an actual network service
<lifeless> mwhudson: thats quite an achievement
<jml> mwhudson: no, I didn't.
<jml> mwhudson: I didn't even know it *had* its own test runner
<wgrant> Django manages to be pretty terrible.
<jml> lifeless: you don't mind having multiple language implementations for a protocol in one tree :P
<wgrant> Without the few redeeming qualities that Zope has.
<mwhudson> lifeless: it rips apart testsuites in the same way zope's does, and has database management baked into the runner directly
<jml> eww
<lifeless> jml: I don't, but its been pretty awkward
<mwhudson> i may try to convince them this is a bad idea
 * jml nods
<lifeless> jml: and I'm a bottleneck on changes in that project, something I want to structurally avoid for our services
<jml> lifeless: ok. I think that's fair enough.
<lifeless> jml: another facet here
<lifeless> jml: we're aiming for 'crisp, clean and strongly defined layers'
<jml> lifeless: the main thing *I* want to avoid is perfect defeating good. Would rather work towards correctness from a starting point of functionality than from where we are now.
<lifeless> jml: anything whose contract is in python is pretty poor at that :)
<jml> lifeless: good point.
<lifeless> jml: have you looked at my draft microservice for gpg signature checking ?
<jml> lifeless: no, but your email about it is flagged for reading
<lifeless> jml: I don't mind the path taken to reach correctness! I mentioned these things so you know where I want us to aim.
<jml> lifeless: cool.
<lifeless> jml: I *do* mind if we try to go live with something without either reaching these things or having and explicit ack from me to waive them in this case.
<lifeless> jml: the point about importing stuff, is that if the rabbit fixture is moved into this other-server-tree, it becomes unusable by Launchpad
<jml> multiplying tiny projects :(
<lifeless> yes
<lifeless> so we put them all in ~launchpad/+junk
<lifeless> this is something LP needs to be much better at
<jml> well, LP doesn't even begin to help you answer the question "how the hell do I land stuff on this project"
<jml> lifeless: I think the 'not importing' requirement should be made explicit in that doc, btw.
<lifeless> wgrant: if there is more than long polling in the tree, exercise rm :)
<wgrant> lifeless: s/rm/split/, I suspect.
<wgrant> lifeless: We need to analyse tomorrow.
<lifeless> wgrant: sure
<lifeless> jml: I will transscribe this discussion in some form today
<wgrant> I didn't look at the code until I was somewhat impaired.
<lifeless> jml: after I get my slides ready
<wgrant> Today was just spent getting it working.
<jml> lifeless: thanks.
<lifeless> jml: I am open to negotation on everything, but I think I have a cohesive story with interlocking supports
<jml> wgrant: and, (I do hate sounding like a broken record here), it's not actually working yet, insofar as it isn't doing what we want.
<wgrant> jml: Sure.
<wgrant> Getting it working, not succeeding in that endeavour :)
<jml> :)
<jml> I'm off to catch some Zs
<jml> g'night.
#launchpad-dev 2011-06-28
<LPCIBot> Project parallel-test build #76: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/parallel-test/76/
<LPCIBot> Project devel build #842: FAILURE in 5 hr 32 min: https://lpci.wedontsleep.org/job/devel/842/
<LPCIBot> Project parallel-test build #77: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/parallel-test/77/
<LPCIBot> Project db-devel build #673: FAILURE in 5 hr 38 min: https://lpci.wedontsleep.org/job/db-devel/673/
<LPCIBot> Project devel build #843: STILL FAILING in 5 hr 16 min: https://lpci.wedontsleep.org/job/devel/843/
<huwshimi> A couple of UI changes to be reviewed if anyone wants to do them:
<huwshimi> https://code.launchpad.net/~huwshimi/launchpad/what-next-712259/+merge/66030
<huwshimi> https://code.launchpad.net/~huwshimi/launchpad/table-headings-728187/+merge/66008
<wgrant> bigjools: I ripped Storm out of lazr.amqp last night.
<bigjools> \o/
<wgrant> And got it working with the rabbit fixture instead of system rabbit with hacked landscape setup script.
<wgrant> But it's slow.
<bigjools> \o/ and :(
<wgrant> So I need to port the test suite to testtools instead.
<wgrant> Rather than using zope.testrunner.
<wgrant> But lifeless and jml know about that.
<bigjools> jfdi
<wgrant> So it's doable this morning.
<wgrant> Rabbit changes aren't landed yet, but the suite passes locally with them.
 * StevenK giggles at "\o/ and :("
<wgrant> So you can happily turn off your system rabbit soon :)
<bigjools> WOOHOO!
<jml> wgrant: ok. I can work with you on that, since my talk is during this plenary session
<jml> matsubara: you should talk to flacoste & deryck about their acceptance test plans
<jml> matsubara: involves selenium & LEPs
<wgrant> jml: I think it should be pretty easy.
<jml> wgrant: yeah
<wgrant> jml: I started it yesterday but then abandoned it when I looked at docs.
<wgrant> I wonder how trivial a rabbit backport to lucid/maverick is.
<wgrant> Or possibly just skip the reject test on older rabbits.
<jelmer> wgrant: new deployment, w00t :)
<wgrant> jelmer: Yup, and the upgrades work fine.
<wgrant> jelmer: Not sure about the automatic Invalid stuff, though.
<wgrant> Thanks for fixing that up!
<matsubara> jml, will do. thanks for the heads up
<LPCIBot> Project parallel-test build #78: STILL FAILING in 1 hr 6 min: https://lpci.wedontsleep.org/job/parallel-test/78/
<wgrant> There we go.
<wgrant> I tried persisting the rabbit process and just recreating the vhost between tests, but that seems to take 100-200ms... but it seems OK to just manually clear out the vhost over AMQP.
<wgrant> So I now have a fast test suite using the rabbit fixture.
<wgrant> yay.
<rvba> wgrant: Could you please update DF when you get a chance?
<StevenK> Wait until I'm done with it.
<StevenK> Please. :-)
<matsubara> Could someone review this branch for me: https://code.launchpad.net/~matsubara/launchpad/724727-single-line-inline-editor/+merge/66089? It's a small css change
<wgrant> StevenK: (too late)
<wgrant> But I didn't restart the appserver, since I know you're using the librarian.
<StevenK> Right
<StevenK> Can you prod and see if memcached is running?
<wgrant> There is a memcached running.
<wgrant> (zomg, no latency!)
<StevenK> 2011-06-28 09:15:04 WARNING Memcache set failed for populate-bprc
<StevenK> :-(
<wgrant> Checking conifg.s
<wgrant> ah.
<wgrant> StevenK: The system memcached is running, but the Launchpad one is not.
<wgrant> Should I start it? (needs appsever/librarian restart)
<StevenK> Wait until garbo finishes
<wgrant> k
<StevenK> 2011-06-28 09:17:56 DEBUG2  [PopulateBinaryPackageReleaseContents] Iteration 1106 (size 1.0): 3.701 seconds
<lifeless> morning guys
<wgrant> Good day sir.
<lifeless> I have my webcam ready to roll
<lifeless> if someone can skype me that would be great
<wgrant> Excellent.
<StevenK> 2011-06-28 09:23:24 DEBUG2  [PopulateBinaryPackageReleaseContents] Done. 1192 items in 1192 iterations, 3267.929582 seconds, average size 1.000000 (0.364756941707/s)
<StevenK> wgrant: Please to be restarting.
<lifeless> spm: http://onefte.com/2011/06/27/youre-such-an-enabler/ is the bomb!
<wgrant> StevenK: Should be running now.
<jml> lifeless: technology ensues
<lifeless> \o.
<jml> lifeless: inappropriate branding!
<lifeless> jml: save-as branding
<StevenK> lifeless: Can we troll you instead?
<StevenK> lifeless: Seriously, please talk a little slower.
<wgrant> It's not that bad...
<lifeless> call dropped
<lifeless> or held ?
<wgrant> Francis killed it.
<wgrant> Somehow.
<jml> lifeless: flacoste pressed the wrong button
<jml> I bet he hit <space>
<bigjools> PEBKAC
<jml> lifeless: that means "99%ile is twice as fast"?
<jml> https://dev.launchpad.net/ArchitectureGuide/Services
<StevenK> lifeless: Does it suffer from the /tmp issue?
<StevenK> \o/
<poolie> lifeless: there's going to be more than 86 lines of work in getting it deployed, is my impression
<jml> poolie: hah!
<poolie> for example the requirement for monitoring etc
<poolie> lifeless: i like the way you've kept banging the drum of performance tuesday
<poolie> lifeless: it might be easier to put questions on irc
<poolie> jml: ?
<jml> lifeless: <wallyworld> we've chosen web.py as our web infrastructure for our microservices, we've had other suggestions here that might be better (e.g. pyramid), why have we chosen web.py?
<jml> poolie: the "more than 85 lines of code" comment
<jml> poolie: it's true.
<poolie> ah
<poolie> i hoped for a moment you were laughing because i was wrong :)
<wgrant> poolie: Well, I think that initial cost is high, but it will become tiny once we have a few services set up similalr.y
<jml> wgrant: yeah, that's what I was thinking
<jml> poolie: maybe one day you will be :)
<poolie> yeah, and as IS continue their projects for puppet, or as things become more containerized
<wgrant> Like haproxy is now.
<wgrant> Initial cost was high.
<wgrant> But now we have it set up, we can add new stuff to it trivially.
<jml> also as we figure out our monitoring needs & come up with stuff that is easy to use, for example
<wgrant> Right.
<poolie> lifeless: this seems a bit inconsistent with saying that we can't use graphite because of (what was it) mental load
<wgrant> It's going to be a slow start.
<jml> although very much agree w/ what lifeless is saying right now about not converging on tech choices too soon.
<poolie> me too
<jml> what are you banging?
<wgrant> I think pyramid is similarly inappropriate.
<wgrant> It seems to be an application framework.
<wgrant> Smaller, but still an application framework, sort of.
<jml> https://dev.launchpad.net/ArchitectureGuide/ServicesRequirements
<jml> for those who want to read them
<poolie> i do agree with experimenting with new technologies, and that new ones are needed
<wgrant> Thanks jml
<stub> I was considering hacking up a version of Robert's work to use ZeroMQ so a direct comparison can be made.
<wgrant> lifeless: But don't the LOSAs already manage graphite instances, so it's not quite the same thing?
<poolie> i guess i find it hard to believe having two graphing services will cause more mental load than having some code in node and same in python
<poolie> nm
<poolie> i don't want to side track
<poolie> the general issue is: what's the amount of diversity that is acceptable?
<poolie> any amount?
<poolie> i'm not trying to debate the particular case here
<jml> so the problem is that it would violate the need for a canonical, trusted, operational data source
<jml> rather than that it would be new tech?
<poolie> lifeless: we're hanging up on you
<jml> lifeless: you're still on screen, fwiw
<poolie> lifeless: i don't mind changing this to tuolumne (much) but i don't understand the architectural position behind it
<poolie> well... i can understand saying it's easier to do them in two steps
<lifeless> poolie: graphing is a core realtime troubleshooting service
<lifeless> poolie: development of a service is a nonrealtime isolated service
<lifeless> poolie: it isn't an architectural position to say that the troubleshooting stuff - logs, graphs, etc - need to be as consistent and reliable as we can
<lifeless> poolie: its a pure ops position
<poolie> don't frameworks have big ops consequences?
<poolie> well, perhaps microframeworks don't, but deploying say go or nodejs seems likely to substantially impact operational debugging
<poolie> well
<lifeless> poolie: there will be an impact yes; its not free
<lifeless> poolie: but by their nature they are more isolated than graphing
<lifeless> bigjools: so, around
<lifeless> ?
<poolie> it's a coffee break now
<poolie> i' guess he'll be back later
<lifeless> poolie: another way to talk about the tuolumne/graphite thing
<lifeless> poolie: for a given microservice, we'll only have *one* implementation live
<lifeless> poolie: (unless we're deliberately migrating [as a funded driven project, not an itch-scratch that could stop at any time] betweeen implementations)
<poolie> that's a reasonable point
<poolie> sinzui, when is that bugfix likely to go out onto qas?
<lifeless> poolie: I can tell you are confused; you feel its inconsistent : I don't think it is, but IRC late at night isn't a brilliant way to tease it out. if you want voice, I can happily do that.
<poolie> it's not a practical problem for me at the moment
<poolie> i'm sorry it apparently took over your session, which wasn't my intention
<lifeless> no worries; it was only a slight segue
<poolie> could someone send me the page performance report url?
<lifeless> poolie: https://devpad.canonical.com/~lpqateam/ppr/lpnet/latest-daily-partition.html
<poolie> the stats stuff i've been reading reinforced to me that its treatment of times as normally distributed is very bogus
<lifeless> hugely
<lifeless> patches solicited!
<poolie> i like the way it has highly optimized algorithms from Knuth implementing the wrong solution
<poolie> complete with citations of learned papers
<poolie> anyhow, go and sleep
<lifeless> poolie: well, for the purpose of giving us -a- baseline and running ok, its fit for purpose
<poolie> right
<poolie> as it speeds up the numbers will go down so it may be mostly academic
<lifeless> poolie: I would be worried if I thought the reported 99th percentile would be low.
<lifeless> poolie: I don't think it will be
<matsubara> benji, https://code.launchpad.net/~matsubara/launchpad/724727-single-line-inline-editor/+merge/66089
<lifeless> bigjools: ping
<bigjools> lifeless: you punged?
<lifeless> I did
<lifeless> do you have a few minutes for a private skype call ?
<bigjools> lifeless: for you, of course, let me get set up, one min
<bigjools> lifeless: actually, 5 mins, I need to run to my room to get my headset
<lifeless> thats fine
<lifeless> I shall go talk to lynne for a minute ;)
<bigjools> lifeless: ok one minute again, allenap lent me his
<huwshimi> Anyone want to do some ui reviews for me?
<huwshimi> More specifically: https://code.launchpad.net/~huwshimi/launchpad/what-next-712259/+merge/66030 and https://code.launchpad.net/~huwshimi/launchpad/table-headings-728187/+merge/66008
<bigjools> lifeless: hang on
<bigjools> headset fail
<lifeless> bigjools: hanging :>
<bigjools> as in skype doesn't want to use it
<lifeless> poolie: deployment overhead for microservices needs automation, but they should be (roughly) one init script [cookie cutter], one log feeder into tuolumne (standard apache format, so no coding), one nagios probe to hit a good url (also cookie cutter), and a stock haproxy config (also cookie cutter)
<lifeless> jml: I haven't written up the testing/importing aspects of services yet, today was more fragmented than I anticipated
<lifeless> stub: a 0mq service impl would be interesting indeed
<lifeless> stub: I am not convinced by my own analysis about ha needs yet ;) - having some experience and learning there would be most excellent
<poolie> i wonder about the 99th percentile; it seems it ought to be possible to calculate it mathematically
<poolie> i wonder if my R is up to it
<barry> gary_poster: when's a good time to get together with you and sinzui?
<lifeless> poolie: one thing to note is our dataset size - we're dealing with 12M rows per day, and for each row we have pageid, service time, sql time, sql query count
<poolie> so any patch needs to not break the scaling behaviour
<bac> has anyone seen abentley?
<huwshimi> bac: Just saw him walk out of the main LP room
<bac> thanks huwshimi...hopefully he's coming upstairs
<huwshimi> benji: Do you feel like doing a couple of reviews for me?
<jml> https://code.launchpad.net/~jml/lazr.amqp/license-and-copyright/+merge/66113
<stub> lifeless: where does the updated batching code live? Looking at Bug #739052   with abel.
<_mup_> Bug #739052: Distribution:+builds timeout <timeout> <Launchpad itself:Triaged by adeuring> < https://launchpad.net/bugs/739052 >
<stub> lifeless: Batching over nearly 800,000 results in a bad case
<stub> lifeless: nm. Found it.
<gary_poster> hey barry!  how about right after lunch, 2-ish?
<barry> gary_poster: sounds good
<barry> gary_poster: i'll meet you in the lp room
<gary_poster> thanks barry!
<poolie> lifeless: fyi mean+3*stddev does underestimate the 99th percentile time of an exponential distribution
<poolie> by about 12%
<StevenK> lifeless: Are you around by any chance?
<lifeless> StevenK: yes
<poolie> this is based just on the generic distributions, not launchpad's specific data
<lifeless> stub: you're going to finish it off \o/
<lifeless> ?
<poolie> it might actually more likely be an erlang distribution, not exponential
<lifeless> stub: there were only a few remaining glitches in my branch, I think.
<lifeless> stub: https://code.launchpad.net/~lifeless/launchpad/bug-752153/+merge/56505
<StevenK> lifeless: Do you know how large the path table for the conflicts checker is?
<lifeless> stub: https://code.launchpad.net/~lifeless/launchpad/bug-759467/+merge/58262 is also -so-close-to-done- that will help with some timeouts
<lifeless> StevenK: not offhand
<jcsackett_> wallyworld: some code supposedly using the App Framework i have just found. https://github.com/ericf/photosnear.me/blob/master/public/photosnearme.js
<poolie> so it's not collossally off, probably
<jcsackett_> er, wallyworld_ ^
<allenap> wgrant, gmb, jml, bigjools: The Rabbit fixture gives off working-like noises and is in lp:~allenap/+junk/lazr.fixtures
<allenap> rvba: Oops, you too ^
<stub> lifeless: Bah. Thought that stuff all landed.
<jml> allenap: cool!
<lifeless> StevenK: the schema:
<lifeless> CREATE TABLE binpackagefiles (package_id INTEGER, file_id INTEGER);
<lifeless> CREATE TABLE filepath (id INTEGER PRIMARY KEY, path VARCHAR);
<lifeless> CREATE TABLE packageversion (id INTEGER PRIMARY KEY, name_id INTEGER, version VARCHAR, arch_id INTEGER, controltext VARCHAR, analyzed_level INTEGER, preinst_hash VARCHAR);
<lifeless>  select count(*) from binpackagefiles;
<lifeless> 49416355
<StevenK> lifeless: Right, I'm curious how large the filepath table is
<lifeless> its counting now
<lifeless> it wasn't running, so its all cold IO
<StevenK> ight
<StevenK> *Right
<lifeless> but clearly < 49M :P
<jcsackett_> wallyworld_: https://github.com/yui/yui3/tree/master/src/app/js
<lifeless> StevenK: select count(*) from filepath;
<lifeless> 11018381
<lifeless> StevenK: do you want the rows?
<StevenK> So 11 million or so
<lifeless>  select * from filepath limit 1;
<lifeless> 1|usr/bin/2vcard
<StevenK> I'm trying to work out how large the BPP and BPRC tables are going to be.
<StevenK> BPP might only be a couple of GiB
<lifeless> average length 61.3
<lifeless> select sum(length(path)) from filepath;
<lifeless> 675274107
<danilos> gmb, hi, I am about to remove your JS code for "collapsibles" which I can't find being used anywhere; you've got 0.04s to complain ;)
<StevenK> lifeless: With 350,000 paths on DF, the table is currently 75MiB.
<lifeless> (e.g. < 700MB)
<gmb> danilos: No complaints. Kill it.
<danilos> gmb, cool, thanks
<poolie> after a branch lands to devel, how soon can i expect to see it on qas?
<lifeless> poolie: 7 hours
<benji> Riddell: http://docs.python.org/library/pdb.html#debugger-commands
<lifeless> poolie: up to 13 if it just missed a buildbot run
<LPCIBot> Project db-devel build #674: STILL FAILING in 5 hr 51 min: https://lpci.wedontsleep.org/job/db-devel/674/
<lifeless> poolie: more if trunk is broken
<lifeless> stub: I wish they had landed :)
<StevenK> stub: Where are you hiding?
<poolie> thanks
<poolie> stevenk also told me to look at the estimate in builbdot and add 30m
<StevenK> lifeless: I question your 7. I thought qas took ~ 30 minutes to update
<poolie> the step up in https://lpstats.canonical.com/graphs/CodebrowseHTTPResponses/ is a bit interesting
<lifeless> StevenK: 6 hours to go through BB
<stub> StevenK: Up with Julian for rabbity stuff
<lifeless> StevenK: 30 minute frequency of qas updates
<lifeless> StevenK: 15 minutes latency for the carob copy to update after BB
<lifeless> StevenK: and then the build-and-deploy time on asuka
<stub> adeuring: I'll send lp:~lifeless/launchpad/bug-752153 off to ec2 test to see how close it is to landing. I think we need this branch to land before we can tackle the batching bug.
<benji> Riddell: http://docs.zope.org/zope2/zope2book/AppendixC.html
<adeuring> stub: ok
<poolie> so probably no less than 7 and possibly more?
<stub> lifeless: So StevenK has about 30GB of new data to store (every file path in every package). Microservice or main LP DB?
<lifeless> poolie: right
<benji> jcsackett_: did you figure anything out on bug 436247?
<_mup_> Bug #436247: links in side portlets are too close <easy> <lp-bugs> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/436247 >
<jcsackett_> benji: i wasn't able to pursue it with my computer randomly dying. :-P
<jcsackett_> benji: as huwshimi pointed out, probably worth bugging sinzui
<lifeless> stub: If a microservice makes sense for it, \o/.
<lifeless> stub: be sure to consider the whole use case in assessing that
<matsubara> bigjools, is there an easy way to create a private ppa in launchpad.dev with some dummy packages in it?
<stub> lifeless: I was poking him to talk to you about it if you are in your teddy bear suit.
<lifeless> sure, I can do that
<lifeless> jml: can you champion my service fake tests etc today, I promise to write it up tomorrow
<lifeless> StevenK: so did you want to talk about doing amicroservice? skype?
<jml> lifeless: ok
<StevenK> lifeless: I haz no skype
<lifeless> StevenK: voip?
<StevenK> lifeless: If it's 30GiB, I'm comfortable doing it in the main database since it will be running as part of the publisher.
<lifeless> jml: thanks!
<StevenK> lifeless: TBH, I came to Dublin not expecting to do anything like Skype. I can only do Mumble, except that I don't have a headset.
<lifeless> StevenK: the size of the data doesn't particular impact the service-nature of it for me
<lifeless> StevenK: things that make a difference are - is it potentially reusable for other goals?
<StevenK> lifeless: It is not.
<lifeless> StevenK: does it have different HA needs to the rest of the system (e.g. if its a cache we could rebuild it on failure rather than having N copies)
<StevenK> It's a list of contents of binary packages, I can't think of anything aside from contents generation that will find that useful.
<wgrant> StevenK: But, equally, there's no reason to have it in LP :)
<StevenK> But it's *easy* if it is in LP.
<lifeless> StevenK: does it have different IO needs than the rest of the system - e.g. will having it in the main DB sacrifice memory to it whereas on a different system it could be always-cold and we wouldn't care
<StevenK> And since this is a spare-time project, I'm loving the easy.
<wgrant> StevenK: But it's in LP if it is in LP.
<lifeless> does the rest of the system need to know about the implementation of this thing?
<StevenK> The publisher will
<lifeless> (if yes, it shouldn't be a service; if no it could be a service)
<lifeless> StevenK: FWIW I think that the contents.gz thing wouldn't make a good microservice on its own; it would need publishing data included in it. Such redundancy isn't necessarily bad (in fact having a service that handles archives, their internal consistency, etc, including contents.gz, would be awesome)
<lifeless> StevenK: so I'm totally fine with you including it in LP itself, given that we don't have that other service to put this in instead.
<StevenK> lifeless: Okay.
<lifeless> StevenK: it will often be easier (short term) to bundle stuff into the LP tree
<lifeless> StevenK: I'm glad stub suggested making it into a microservice; I think this particular case is probably too awkward right now, but we need to start doing it more
<lifeless> alrighty, its pumpkin time
<lifeless> ciao
<bigjools> matsubara: hey, sorry didn't answer you earlier
<jtv> wgrant, StevenK: do you have uncommitted changes in dogfood's source tree?
<StevenK> Yes.
<jtv> boo
<nhandler> I'm writing a Perl script to try and interact with Launchpad using the API. I am able to get a request and access token, but making signed API requests fails. I get a 401 error with a 'Client-Warning: Missing Authenticate header' (despite having a request matching that of the documentation) and an 'Unknown consumer (nameOfMyApp)' message. Any ideas on what might be wrong?
<jcsackett> wallyworld: https://code.launchpad.net/~launchpad-teal-squad/launchpad/lazr-js-kicking-and-screaming
<rvba> wgrant: could you send me these cryptic python lines that you used to publish a message on rabbit?
<wgrant> rvba, bigjools: http://paste.ubuntu.com/634325/
<huwshimi> benji: https://code.launchpad.net/~huwshimi/launchpad/what-next-712259/+merge/66030 and https://code.launchpad.net/~huwshimi/launchpad/table-headings-728187/+merge/66008
<wgrant> bigjools, rvba: http://paste.ubuntu.com/634326/
 * benji looks.
<jcsackett> huwshimi: https://code.launchpad.net/~launchpad/launchpad/lazr-js-kicking-and-screaming
<wallyworld_> rockstar: ping!
<benji> huwshimi: both branches are approved, I had a thought on the second you might be interested in (or not;)
<huwshimi> benji: Thanks a lot, I'll take a look
<bigjools> stub: http://paste.ubuntu.com/634326/
<rockstar> wallyworld_, pong
<bigjools> rvba: http://pastebin.ubuntu.com/634349/
<wallyworld_> hi rockstar. long story. we are packaging yui as a tarball to go in download-cache and i am porting the lazr-js build utils across to lp. someone said you had done something similar. i was wondering about any gotchas that i should be aware of
<rockstar> wallyworld_, https://launchpad.net/lazr.jstools
<wallyworld_> rockstar: thanks. i'll take a peek
<rockstar> wallyworld_, hm, there seems to be no branch there, but I'm sure I pushed it.
<rockstar> wallyworld_, lemme dig around, I'm sure I have a backup of it as well.
<wallyworld_> rockstar: ok. thanks. appreciate it
<rockstar> wallyworld_, I will say that the tools we were using in lazr-js tend to be less effective than, say YUI Compressor.
<rockstar> U1 is now using YUI compressor.  It compresses YUI code a lot better than Chrome's js tools we were using in lazr-js.
<bigjools> rvba: http://pastebin.ubuntu.com/634351/
<wallyworld_> rockstar: ok. we'll need to take a look at that too then. first step is a straight port and then we can tweak and optimise :-)
<danilos> jtv, https://code.launchpad.net/~danilo/launchpad/branch-portlet-details-remove/+merge/66117
<danilos> poolie, I've added the link to jtv's branch to the project wiki page, fwiw
<poolie> thanks!
<wgrant> rvba: http://paste.ubuntu.com/634369/
<rvba> wgrant: thanks.
<jml> Narwhals! Narwhals!
<wgrant> Swimming in the ocean.
<poolie> Riddell: https://bugs.launchpad.net/bugs/801388
<_mup_> Bug #801388: some person pickers show "assign me"/"remove assignee" when that makes no sense <disclosure> <person-picker> <ui> <Launchpad itself:In Progress by wallyworld> < https://launchpad.net/bugs/801388 >
<jml> jcsackett: http://metaquotes.livejournal.com/6644038.html
<jelmer> jml: Do you perhaps have a moment to go over the BFBIA LEP this week?
<jml> jelmer: yes, I do.
<jml> jelmer: could do it pretty soon, actually
<jml> jelmer: otherwise, make an appointment on my calendar.
<jelmer> jml: Pretty soon works for me, too
<jml> jelmer: ok.
<LPCIBot> Project parallel-test build #79: STILL FAILING in 1 hr 1 min: https://lpci.wedontsleep.org/job/parallel-test/79/
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #844: FIXED in 5 hr 52 min: https://lpci.wedontsleep.org/job/devel/844/
<danilos> deryck, hey, since sinzui is not on IRC, how about you look at https://code.launchpad.net/~danilo/launchpad/speed-up-subscription-tests/+merge/66120 :)
<deryck> danilos, sure.  :-)  Looking now....
<deryck> danilos, Y.lazr.anim was already available due to YUI().use for the test module?
<danilos> deryck, yeah, and fwiw, the test works
<danilos> deryck, I also increased the wait to 200ms from 20ms on sinzui's suggestion
<deryck> danilos, I'm worried the 200ms is still too long.  But there can be some difference between machines....
<deryck> danilos, but I would think 100ms should be enough even for slower machines.
<deryck> unless the dom work is happening beyond the anim completing.
<deryck> danilos, but sinzui tells me know 100-200ms is expected.  So I say try 100.  Lower is always better :)
<danilos> deryck, 20ms works for me just fine, but let's try 100ms for buildbot (if it ever gets back there :)
<deryck> danilos, I'm chatting more with sinzui about this now. his machine is what failed earlier, not buildbot.
<danilos> deryck, oh
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #675: FIXED in 5 hr 53 min: https://lpci.wedontsleep.org/job/db-devel/675/
<lifeless> flacoste: hi
<LPCIBot> Project devel build #845: FAILURE in 5 hr 34 min: https://lpci.wedontsleep.org/job/devel/845/
<poolie> lifeless, Riddell, maybe we should just remove that option
<poolie> does anyone ever want to see the least recently changed bugs?
<poolie> i suppose maybe if you want to update the stalest inprogress or incomplete bugs
 * jelmer doesn't think he's ever used it or would ever use it
<poolie> perhaps 'changed long ago'
<poolie> kind of gets into whether people ven know what 'changed' means, exactly
<poolie> the bug in this area i would actuall ymost like closed is https://bugs.launchpad.net/launchpad/+bug/277352
<_mup_> Bug #277352: should be easier to search for closed bugs <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/277352 >
<poolie> (which is not so trivial, but maybe not so hard)
<poolie> i'm surprised it doesn't have more dupes
<poolie> lifeless: when's the next lpnet rollout?
<jelmer> poolie, that one has annoyed me too in the past
<lifeless> poolie: when someone in dubling requests it
<poolie> is it cheap? can i just request it?
<lifeless> if you follow the process, yes.
<lifeless> one bit of which is 'do not deploy right before everyone goes to sleep
<lifeless> normally thats friday afternoons, but during sprints, any deploy except first thing in the morning would qualify ;)
<wgrant> poolie: There's also nothing to deploy right now.
<wgrant> Lots of QA pending.
<wgrant> rvb/abentley/jtv/matsubara
<poolie> :)
<poolie> ah this'd be https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html ?
<wgrant> Yes
<poolie> wow, that's quite a queue
<poolie> thanks
<lifeless> we can deploy any qa
<lifeless> 'd prefix of it
<lifeless> wgrant: courier rang, new machine arrives in the next 4 hours
<wgrant> lifeless: !
<wgrant> poolie: We deployed 30 revs yesterday.
<poolie> nice
<wgrant> No.
<wgrant> Because the queue was huge.
<wgrant> And still is.
<poolie> lifeless: what model was it?
<poolie> s//is
<lifeless> poolie: dell aurora r3 4(8) core w/16GB
<lifeless> the cpu and mb should handle 32gb but you cannot source the dimms at the moment
<poolie> wow, overclocked from the factory?
<lifeless> wgrant: so I know its not finished, but I need your aufs notes ;)
<wgrant> lifeless: Let me see if WOL works.
<wgrant> It does.
<lifeless> wgrant: + your testr config etc etc.  :>
<wgrant> http://paste.ubuntu.com/634628/
<wgrant> http://paste.ubuntu.com/634629/
<wgrant> Then 'TEMP=/tmp/testr testr run --parallel'
<wgrant> (yes, the /tmp/testr hack is terrible, but meh, it works)
<lifeless> I'll deal
<wgrant> This expects the lucid-lp-base container to have had rocketfuel-setup, utilities/launchpad-database-setup and make schema run.
<wgrant> I'm in no fit state to experiment further now, but if you have any issues I can try to recall now or exeriment tomorow.
<lifeless> well
<lifeless> I'm going to go make sure I have a natty boot cd
<lifeless> I expect backing up the windows install and fiddling around data migration etc will take a chunk of time today
<wgrant> Yeah.
<wgrant> You forgot the ritual zeroing of the disk to ensure obliteration of the Windos installation.
<wgrant> Most of the problems I encountered were around dbus/avahi issues, which are irrelevant now I extract the last DHCP lease.
#launchpad-dev 2011-06-29
<wgrant> Just make sure lucid-lp-base isn't running while the clones are.
<wgrant> Doesn't matter wth the btrfs variant, but for aufs it does.
<wgrant> Also, it needs to have your SSH key in authorized_keys, as you may be able to tell.
<wgrant> There's probably some huge piece I've completely forgotten, but hunting for that can happen tomorrow.
<LPCIBot> Project db-devel build #676: FAILURE in 5 hr 48 min: https://lpci.wedontsleep.org/job/db-devel/676/
<lifeless> wgrant: well, I am probably going to keep 100GB or so of windows around Just In Case
<lifeless> I'm assuming dmraid will handle the dell raid setup
<wgrant> Bah.
<wgrant> Possibly.
<lifeless> mwhudson: you have a sandybridge machine right ?
<mwhudson> lifeless: i do
<lifeless> did you have any issues with the natty desktop cd ?
<lifeless> I get the fine grey crosshatch and nothing more
<mwhudson> lifeless: no, installation went absurdly smoothly
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #846: FIXED in 5 hr 29 min: https://lpci.wedontsleep.org/job/devel/846/
<lifeless> mwhudson: interestingly, its happy with lcd, but it hated my analogue monitor.
<mwhudson> lifeless: heh
<lifeless> 50% of my desktop backed up
<mwhudson> lifeless: i've tested mine with a projector over vga and that worked
<mwhudson> (post-install)
<lifeless> mwhudson: this has a non-intel graphics card in it, for additional fun
<lifeless> being a desktop and all
<mwhudson> ah ok
<mwhudson> well i can't help you with that then :)
<lifeless> :)
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #677: FIXED in 5 hr 51 min: https://lpci.wedontsleep.org/job/db-devel/677/
<lifeless> jml: https://dev.launchpad.net/ArchitectureGuide/ServicesRequirements#preview
<wgrant> bigjools: Any objections to my comments on bug #797599?
<_mup_> Bug #797599: Copying archivepermissions cross-distro is wrong. <derivation> <qa-untestable> <Launchpad itself:Fix Committed by rvb> < https://launchpad.net/bugs/797599 >
<bigjools> wgrant: I don't see any comments from you
<wgrant> bigjools: Packagesets are per series.
<wgrant> bigjools: So after duplicating the sets we need to regrant permissions.
<StevenK> Why did that just get removed? It's trivial to fix.
<wgrant> launchpad_dev=# \d packageset
<wgrant> [...]
<wgrant>  distroseries    | integer                     | not null
<wgrant> abentley: You has QA.
<bigjools> wgrant: RARGH
<StevenK> Haha
<StevenK> That's my line.
<wgrant> bbiRargh?
<wgrant> Er.
<wgrant> bigjools: Rargh?
<bigjools> having to re-init *archive* level permissions on a new series is crack
<wgrant> Well.
<bigjools> the model is broken
<wgrant> Yes.
<wgrant> Sort of.
<wgrant> Packagesets have to be per-series.
<bigjools> I know
<wgrant> So permissions based on them have to be.
<bigjools> the FK is wrong
<wgrant> How?
<wgrant> Well, how would it be improved?
<bigjools> NFI ATM
<wgrant> Besides nuking from orbit.
<wgrant> Right.
<wgrant> Exactly.
<bigjools> need to think about it
<bigjools> but not in a plenary!
<wgrant> True.
<wgrant> I won't roll it back for now, but will file a critical regression bug.
<bigjools> that's fine, we can fix it
<bigjools> next week
<lifeless> morning d'blin
<wgrant> Hi lifeless.
<lifeless> apparently dmraid doth not split IO across underlying devices
<lifeless> this saddens me
<wgrant> Yay!
<wgrant> Fakeraid must die.
<StevenK> wgrant: Including MD?
<lifeless> wgrant: its not substantially different to mdraid other than needing to match BIOS supported layouts, *and* being able to boot sanely.
<StevenK> And wgrant has the deployment report open. Who would have thunk it.
<lifeless> wgrant: i like being able to boot sanely.
<StevenK> lifeless: Er? GRUB2 can boot off RAID
<lifeless> StevenK: last I saw it requires the device to be on a single disk, because its not actually bringing up all the layers
<lifeless> its essentially running in degraded form
<StevenK> lifeless: My fileserver boots off RAID1 just fine
<lifeless> this means it won't work for anything other than a vanilla mirror
<StevenK> But I can't check, since my home DSL is down.
<lifeless> StevenK: e.g. vanilla mirror
<StevenK> You're closer, so go check. :-P
<lifeless> StevenK: ring your wife :)
<wgrant> StevenK: MD is not fakeraid.
<wgrant> It's software RAID.
<lifeless> wgrant: so is fakeraid
<wgrant> It is, but with an extra layer of proprietary crap.
<lifeless> wgrant: its really not that proprietary.
<lifeless> crappy docs though
<wgrant> Stupid and pointless and non-standard, then.
<lifeless> wgrant: what would make it a standard?
<wgrant> Not being chipset-dependent.
<StevenK> It mainly exists because versions of Windows didn't do RAID, right?
<lifeless> StevenK: *booting* from RAID requires either fragile gymnastics or a BIOS (or lower) support for the raid layout
<lifeless> StevenK: the ICH (and similar) raid implementations exist because controller cards that have ATAPI/SCSI on the topside and N disks on the bottom side are hugely expensive for entry-level environments, and *other* than such cards ... you had to do gymnastics
<lifeless> so fakeraid does *exactly* what the early boot BIOS interrupt calls do for SCSI and IDE disks, and run just long enough to let the OS load its drivers.
<lifeless> if fakeraid is stupid, so is being able to boot off of IDE and SCSI disks.
<gmb> wgrant: Is it possible to set append-revisions-only on an existing branch? I can only find references to it on bzr init.
<wgrant> gmb: You probably have to edit .bzr/branch/branch.conf using something like hitchhiker.
<wgrant> bzr reconfigure doesn't seem to have an option.
<gmb> wgrant: Ah, right, thanks.
<wgrant> Just add a line with 'append_revisions_only = True' to branch.conf
<maxb> You can use bzr config now to set append_revisions_only
<wgrant> Ahh, true.
<wgrant> Forgot about the new shiny.
<StevenK> LIFELESS!
<StevenK> RARGH
<wgrant> StevenK: Oh?
<StevenK> parellel-test #80 has been building for *13* hours
<wgrant> Heh.
<StevenK> hudson   25142  0.0  0.0      0     0 ?        Z    Jun28   0:14 [/usr/bin/python] <defunct>
<StevenK> ORSUM
<StevenK> wgrant: Shall I kill it with fire?
<lifeless> jml: hi
<lifeless> jml: I've put some prose into the servicesrequirements page
<jml> lifeless: yeah, I saw, thanks.
<jml> lifeless: will look soon.
<lifeless> I think it captures all that we spoke about, and includes sufficient rationale
<lifeless> jml: if it doesn't, please fix-or-tell-me
<lifeless> ?
<nigelb> StevenK: heh, weren't you about to kill all of windmill with fire? :)
<nigelb> *windmill tests
<LPCIBot> Project parallel-test build #80: ABORTED in 12 hr: https://lpci.wedontsleep.org/job/parallel-test/80/
<StevenK> DIE, FIEND
<nigelb> lol
<jml> lifeless: ok, will do.
<StevenK> nigelb: parallel-test has nothing to do with Windmill.
<nigelb> StevenK: ah!
<nigelb> One of my merge requests has become sort of stale.  Before I work on that branch, I want to "rebase" it wth master
<nigelb> is there a bzr equivalent of that?
<nigelb> (this is an LP merge proporsal that I need to finish working on)
<StevenK> Don't rebase, just bzr merge devel in
<jml> lifeless: any ideas for useful LP (or related) hacking for me this week?
<wgrant> lifeless: :( resetting rabbit is hard.
<lifeless> jml: just about anything is useful
<StevenK> wgrant: Do you think there is any point to keeping the windmill-{db-,}devel jobs on Jenkins, or shall I delete them?
<wgrant> StevenK: I'd keep them for now, just in case. It is cheap...
<lifeless> jml: one thing that you and I probably have most experience around is getting the test suite to stop stomping on stdout when something goes wrong
<jml> lifeless: yeah, I know. that's largely the problem :D
<lifeless> jml: that messes up subunit
<nigelb> er, I did "bzr merge ../devel" and now it seems to have just gotten stuck
<jml> lifeless: yeah, ok, I could try with that.
<StevenK> nigelb: Is devel up-to-date?
<nigelb> doesn't merge give me feedback on stdout on what's going on?
<nigelb> StevenK: yeah
<nigelb> oh
<nigelb> done :D
<StevenK> Heh
<stub> I always wanted to make tests fail that spat stuff out to stdout or stderr
<StevenK> More patience required
<stub> Not sure the most bulletproof method of doing that though
<jml> stub: interestingly, the way we patch the login object practically guarantees that tests will
<nigelb> Sadly, I can't apt-get install patience :)
<StevenK> I'd like to see an option to bin/test that will stop spewing the librarian log out on test failure
<lifeless> deprecation warnings and bzr's __del__ methods are particularly hard to get at to fix this
<lifeless> StevenK: its an attachment to the test
<lifeless> StevenK: something you could do that is better
<lifeless> StevenK: would be to teach the layer to only attach the *per-test* component of the log
<lifeless> StevenK: rather than the log-since-fixture-start.
<lifeless> StevenK: e.g. truncate the log or record the position or whatever in the layer per test setup
<lifeless> StevenK: if you use tribunal though,t he librarian log won't be in your face anyway
<spiv> lifeless: most of bzr's __del__ methods are being deleted.
<lifeless> spiv: I know :)
<lifeless> spiv: (and I'm glad)
<StevenK> lifeless: Bleh, more reason to learn tribunal
<nigelb> wow, I updated my devel again.
<nigelb> Lots of deletions \o/
<StevenK> % apt-cache search tribunal | wc -l
<StevenK> 0
<nigelb> I guess that's the windmill being burned with fire.
<StevenK> jml killed lots of line too
<lifeless> StevenK: poolie: can get you going with tribunal
<nigelb> I now realzie I'm going to fix the same bug 4 times.
<_mup_> Bug #4: Importing finished po doesn't change progressbar <lp-translations> <Launchpad itself:Fix Released by carlos> <Ubuntu:Invalid> < https://launchpad.net/bugs/4 >
<lifeless> StevenK: it has testr integration, so its basically just tribunal .
<StevenK> I don't have testr either ...
<lifeless> StevenK: that one is packaged; shiniest version is in the testing cabal ppa, but the one in natty is eminently usable
<wgrant> lifeless: It seems there is no way to list all rabbitmq queues without using rabbitmqctl, which is 100-200ms :(
<wgrant> lifeless: Which makes fixture reset difficult.
<lifeless> wgrant: :(
<lifeless> wgrant: there are some alternatives
<wgrant> Creating a new vhost is also 100-200ms.
<wgrant> Seems to be erlang startup.
<lifeless> wgrant: such as, use the in python api to note queue access and connect-and-flush-all-those at the end of the test
<wgrant> lifeless: Yeah, possibly...
<wgrant> lifeless: But that becomes really bad when we have multiple consumers.
<wgrant> That we don't control.
<lifeless> wgrant: for an LP test run, all consumers should be test fakes
<lifeless> wgrant: which means we have a testing interface to them, and can tell them to do their own reset dance
<wgrant> "should"
<lifeless> wgrant: the page has been changed.
<wgrant> Also, I am talking about now.
<wgrant> Not in two years.
<lifeless> wgrant: *must*, or it can't go live. Existing things like the librarian are excluded
<wgrant> Hmm.
<lifeless> I'm flexible on this, but I think that being waffly would make it harder for folk to design well
<lifeless> so I've stopped waffling on this aspect.
<lifeless> by flexible I mean I am willing to be wrong and discuss; however your objection seemed to be that there was no requirement so we would have trouble in the future.
<StevenK> abentley: Can haz QA? And not submitting branches with [r=me] ?
<wgrant> Well, I told him to verbally, but I didn't think he'd take it literally :)
<lifeless> StevenK: do you mean 'please use r=<ownname> rather than r=me when self reviewing' ?
<lifeless> StevenK: if so, +1
<StevenK> r=me or rs=me is so pointless it may as well not be there.
<StevenK> Since loggerhead on devel won't tell us who commited it
<StevenK> Which means you either need to know, or ask buildbot. And asking buildbot makes me sad.
<wgrant> Or check the committer of the merge revs.
<wgrant> lifeless: Hmm, FixtureResource doesn't use reset()?
<wgrant> The rabbitmq management plugin does what we want, but the rabbitmq plugin build system seems to be designed primarily to make people want to die.
<lifeless> ha. headdesk. ha.
<StevenK> 103 files mention 'unseen' -- some would be false-positives
<wgrant> Perhaps we should use carrot.
<wgrant> (it has a memory-based backend, as well as a rabbit one)
<wgrant> Which would allow us to do unit tests with a nice fake.
<lifeless> wgrant: that only helps if we're not talking to a microservice
<Riddell> simple review needed https://code.launchpad.net/~jr/launchpad/797688-packaging-branches-label
<StevenK> Riddell: I'd like to see some test changes, to be honest.
<benji> someone asked about this yesterday and I wan't yet aware that it existed: http://ricostacruz.com/js2coffee/
<lifeless> benji: wicked
<benji> poolie: this just came up on my feeds and wondered if you'd seen it: https://www.mergebox.net/
<benji> they bill themselves as the GitHub of bzr
<lifeless> benji: they announced themselves on the bzr dev list
<lifeless> benji: very interesting project
<benji> ahh, I should have figured they would engauge in some way
<lifeless> not at all
<lifeless> its good to raise interesting things
<wgrant> lifeless: Sure.
<LPCIBot> Project parallel-test build #81: STILL FAILING in 1 hr 11 min: https://lpci.wedontsleep.org/job/parallel-test/81/
<lifeless> usb2 is so slow!
<wgrant> bigjools: It is the 'ppa' name.
<wgrant> bigjools: Not 'Test'
<wgrant> So it's not quite so easy.
<bigjools> yeah just saw
<LPCIBot> Project db-devel build #678: FAILURE in 5 hr 51 min: https://lpci.wedontsleep.org/job/db-devel/678/
<LPCIBot> Project parallel-test build #82: STILL FAILING in 37 min: https://lpci.wedontsleep.org/job/parallel-test/82/
<LPCIBot> Launchpad Patch Queue Manager: [r=jml][no-qa] Remove syncUpdate.
<poolie> benji, lifeless, has anyone actually got a login yet?
<lifeless> I haven't; I can't decide if I should, or should not :)
<benji> heh
<nigelb> poolie: maco did get a login for it if you're looking for feedback
<nigelb> (it being mergebox)
<lifeless> bigjools: dsds are shaping up really quite nice
<bigjools> lifeless: hellyeah!
<lifeless> bigjools: I hit a small usability thing today, not sure if its a bug or not.
<lifeless> bigjools: I wanted to answer, for a friend 'whats different between debian and ubuntu for virt-manager and libvirt-bin'
<bigjools> that is not enough info to properly ascertain an answer
<lifeless> bigjools: localpackagediffs doesn't seem to have a unfiltered view
<bigjools> it's always series-dependent
<bigjools> lifeless: !
<bigjools> yes it does
<lifeless> like there is unignored, ignored, ignored-and-newer, and resolved
<bigjools> oh that
<bigjools> there's a bug about tha
<bigjools> t
<lifeless> kk
<lifeless> I shall not file one then
<lifeless> did I mention usb2 is painfully slow ?
<bigjools> lifeless: thx for feedback though
<lifeless> de nada
<lifeless> my friend uses Debian and then whinges that stuff doesn't Just Work :)
<StevenK> lifeless: USB 2 sucks, use something better.
<bigjools> there's an answer to that
<lifeless> StevenK: would-if-I-could
<StevenK> lifeless: There is this thing, called eSATA?
<lifeless> StevenK: external disk only having usb2
<lifeless> StevenK: I just backed up everything from my old desktop onto it, and it got 30MB/s sustained... restoring at the same rate :(
<StevenK> lifeless: The last time I copied stuff over USB 2, I copied 800GiB at an average of 3MB/s, and it took 4 days.
<lifeless> StevenK: -ouch-
<lifeless> StevenK: I'm only doing 400GB and still its painfule
<huwshimi> StevenK: https://launchpad.net/launchpad/+bugs?field.tag=easy+ui&field.tags_combinator=ALL
<huwshimi> StevenK: I forgot to mention, if you fix a bug then tag it with "dublin" so we can keep track of the bugs fixed here
<huwshimi> benji: I have updated this branch, just wanted to check the fix looks ok: https://code.launchpad.net/~huwshimi/launchpad/table-headings-728187/+merge/66008
<benji> huwshimi: I made positive sounds on the MP.
<huwshimi> benji: Thank you!
<Riddell_> StevenK: are you able to point me in the direction of writing tests for my 797688-packaging-branches-label branch at some point?
<StevenK> benji: https://code.launchpad.net/~stevenk/launchpad/archive-picker-value/+merge/66318
<jtv> bigjools, StevenK, wgrant: any objections to a df app restart?
<StevenK> jtv: No, doit.
<wgrant> jtv: None.
<jtv> StevenK: who are you calling a doit?
<jtv> â¦or whommm?
<StevenK> jtv: You, duh.
<jtv> oh, I see.
<benji> StevenK: done
<danilos> jtv, hey, I've got a question about the expander widget: how do I make it start out expanded?
<jtv> danilos: disable JS.  :)
<danilos> jtv, ok, so that's a feature we are missing: if "expanded" flag is on it on load, we should probably not fold it
<jtv> Sure.
<danilos> jtv, it's a case I hit on a page
<jtv> I mentioned this yesterday as "nice for later"; I guess "later" came early
<danilos> jtv, ok, I'll assume that works and just set the "expanded" class on the content node
<jtv> BTW where is that flag exactly?  Is it a CSS class, a config arg..?
<danilos> jtv, yeah, I was not sure if it was implemented or not, and it's needed to be able to replace the existing collapsible stuff
<jtv> Ah
<danilos> jtv, a CSS class state_marker.expanded
<jtv> OK
<Riddell> StevenK: https://code.launchpad.net/~jr/launchpad/797688-packaging-branches-label/+merge/66146
<bigjools> jtv: nup
<jtv> ?
<Riddell> StevenK: voila
<Riddell> hmm, wait
<StevenK> benji: https://code.launchpad.net/~mbp/lazr.restfulclient/789369-getattr/+merge/62743
<matsubara> anyone available to review a branch that attempts to fix bug 404279? https://code.launchpad.net/~matsubara/launchpad/404279-private-ppa-link/+merge/66335
<_mup_> Bug #404279: apt urls on private ppa pages are incorrect (missing credentials) <bugjam2010> <easy> <lp-soyuz> <p3a> <ppa> <ui> <Launchpad itself:In Progress by matsubara> < https://launchpad.net/bugs/404279 >
<poolie> i will try
<bigjools> stub: addAfterCommitHook doesn't work....
<bigjools> he no callee backee
<stub> You said it worked!
<bigjools> I LIED
<stub> commit is being called?
<bigjools> yes, we just stepped in pdb
<stub> bigjools: So we know the concept is sound, or else PostgreSQL connections would never be committed.
<stub> bigjools: Oh.... aftercommithook... not sure if they are used anywhere.
<bigjools> indeed
<stub> Maybe just a standard commit hook?
<bigjools> define "standard" ?
<stub> We can worry about ordering of rabbit or pg first later
<bigjools> the other one I can see in the doc is  beforeCommitHook()
<bigjools> stub: maybe we need registerSynch
<stub> Now I can't find the documentation...
<bigjools> stub: http://readthedocs.org/docs/zodb/en/latest/transactions.html
<wgrant> It's not a synchronizer.
<stub> You can register a class with the prepare, commit, rollback hooks like a pg database connection does
<wgrant> It's not a beforeCommitHook.
<wgrant> It's also not an IDataManager
<bigjools> stub: how do you register that?
 * stub wonders if our transaction package is out of date with those docs
<bigjools> and buggy
<stub> Writing it as a full blown datamanager should work
<stub> And not too traumatic since most of the methods will be noops
<bigjools> ok
<bigjools> stub: got an example we can see?
<bigjools> preferably in our code :)
<stub> bigjools: Those docs have one. Otherwise ZStorm has one. Nothing in the lp tree though.
<stub> The one in the docs will be simpler, and that api has been stable for over 10 years (apart from doom which we added)
<stub> Hmm... but I wonder if something else is going on. Given the aftercommit api exists, I would sort of expect it to do what it says on the tin
<stub> bigjools: Your hook has the right signature?
<stub> foo(success, args, kwargs)
<stub> kws
<stub> finished here - popping down
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #679: FIXED in 5 hr 52 min: https://lpci.wedontsleep.org/job/db-devel/679/
#launchpad-dev 2011-06-30
<lifeless> so I have my test machine up and running, but boy was it more work than I expected
<wgrant> lifeless: Around?
<lifeless> yes
<lifeless> 'sup?
<lifeless> wgrant: ^
<wgrant> lifeless: I tried to use testresources in lazr.amqp, with ResourcedTestCase. But it cleans and sets up the fixture for every test. Do I have to work in OptimisingTestSuite somehow?
<lifeless> yes
<lifeless> what test runner are you using ?
<wgrant> At the moment it's zope.testrunner, but we're not tied to that.
<wgrant> Will gladly abandon it.
<StevenK> jtv!!!
<wgrant> Still QA?
<jtv> StevenK: ?
<lifeless> wgrant: you will have to
<wgrant> lifeless: Excellent.
<StevenK> jtv: You have 4 or 5 branches of QA!
<lifeless> wgrant: it messes with uites
<wgrant> lifeless: Should I just use unittests?
<jtv> StevenK: no I don't think so
<wgrant> s/unittests/unittest/
<lifeless> wgrant: subunit.run :)
<jtv> StevenK: I had to fix an ancient +queue timeout though before I could do meaningful Q/A
<wgrant> lifeless: Can I integrate that with buildout, or should I just use testr?
<StevenK> jtv: Sorry, 3. r13302, r13318 and r13326
<lifeless> wgrant: I don't know about buildout and tests
<jtv> StevenK: I have zero branches in qa-needstesting.
<jtv> Your view may be lagging.
<StevenK> When did you change the bugs?
<bigjools> pay attention gentlemen
<jtv> StevenK: Sometime in the past 4 minutes.
 * StevenK makes a ... sign at bigjools 
<StevenK> lifeless: I am disabling parallel-test again.
<LPCIBot> Project parallel-test build #83: ABORTED in 11 hr: https://lpci.wedontsleep.org/job/parallel-test/83/
<lifeless> wgrant: chatting with jml for a quick 'how to glue it together' bootstrap may help you
<StevenK> Because of ^
<wgrant> lifeless: Thanks.
<jtv> bigjools: pay attention
<wgrant> lifeless: So we can't use test discovery?
<lifeless> sure can
<lifeless> need the discover egg
<wgrant> How do we make it use OptimisingTestSuite then?
<lifeless> and a root level load_tests hook to wrap things and reenter into discovery to gather the children
<wgrant> k
<wgrant> Will hopefully talk to jml.
<danilos> jtv, hey, the approach of not having a separate expander/content box is not going to work because the spinner still ends up in the expander box
<lifeless> matsubara: hi
<lifeless> matsubara: I just wanted to check you saw my comment on the bug you incompleted
<matsubara> lifeless, hello
<lifeless> matsubara: (because you might not be subscribed ;))
<matsubara> lifeless, I have now. thanks
<lifeless> :)
<matsubara> lifeless, yeah, I didn't get a mail notification :-)
<matsubara> hmm I always thought incomplete could be used as a needs info
<lifeless> if we turn bug expiry off, it could
<lifeless> but bug expiry *ignores* whether a bug has had a reply
<matsubara> right, got that. I won't do that anymore
<lifeless> even if we turned it off, its a bit orthogonal
<lifeless> we over-conflate things in the status field
<matsubara> lifeless, btw, do you have an answer to the question I asked there?
<lifeless> e.g. is-valid, is-bug-or-is-feature-request, is-ready-to-code and filer-has-been-asked-a-question
<lifeless> matsubara: remind me  :)
<matsubara> lifeless, Would it be sufficient to add another bullet point to the +mailinglist page (e.g. https://qastaging.launchpad.net/~oops-tools-dev/+mailinglist) saying something like: "Non-Launchpad users won't be able to post to this list. Launchpad doesn't send non-delivery receipts to non-LP users if they try to post to this mailing list."?
<jml> lifeless: yes, that's what the BugLifecycle thing is partly aiming to address (or diagnose)
<lifeless> jml: yes ;)
<lifeless> matsubara: that would be a good thing to do. I don't think it will eliminate the confusion. It may reduce it.
<lifeless> matsubara: I don't know if it would be sufficient.
<lifeless> matsubara: its probably necessary (unless we change the behaviour, which is problematic for a whole other set of issues)
<matsubara> lifeless, is there other place that documentation could be added?
<lifeless> theres probably stuff on h.l.n about lists in general
<matsubara> lifeless, right.
<matsubara> I'll work on it today
<lifeless> cool
<bigjools> jtv: do you reckon we could make a query fast enough to make +localpackagediffs look for matching packages, packagesets and people associated with packages all at the same time?
<wallyworld_> jcsackett: i've found some issues with non lazr-js build. am fixing now
<jml> rvba: please use a bigger font :)
<jtv> bigjools: we'll have to talk about it â I may not fully understand what it is you want
<rvba> StevenK: qa-ok!
<lifeless> huwshimi: hi
<huwshimi> lifeless: Hey there
<lifeless> so bug 697489
<_mup_> Bug #697489: "Participation" and "Involvement" portlets order is different from "applications" links <easy> <ui> <users> <Launchpad itself:Won't Fix> < https://launchpad.net/bugs/697489 >
<lifeless> huwshimi: how do you want to hae this discussion?
<huwshimi> lifeless: Is here ok?
<lifeless> its fine by me
<lifeless> huwshimi: the getting involved portlet shows up on only one page
<lifeless> huwshimi: I don't see how it can be such a key element
<huwshimi> lifeless:  It appears in various forms on project pages, user profiles, bug listings, bugs, answers etc.
<lifeless> nope
<lifeless> project and person only AFAICT
<lifeless> specifically the overview (root domain) view for pillars
<lifeless> huwshimi: it may have been intended to be widely deployed, but it hasn't been
<huwshimi> lifeless: How do you think most people file a bug?
<lifeless> huwshimi: apport
<lifeless> huwshimi: (statistically)
<NCommander> StevenK: wgrant we should probably meet up tonight to talk about LP
<lifeless> huwshimi: excluding that special case they use the 'report a bug' link which is visible on bugs.l.n/project
<StevenK> NCommander: Tonight, ronight?
<StevenK> s/r/t/
<lifeless> huwshimi: log analysis can tell use if folk start from the project front page, but this involvement portlet is new
<huwshimi> lifeless: Right, so that would seem like a pretty important part of the UI to me
<lifeless> huwshimi: I suspect muscle memory has most folk starting with bugs.l.n/project
<lifeless> huwshimi: thats a different portlet
<NCommander> StevenK: well, its tonight, or tomorrow night, and I find that friday nights are less likely to get something done
<lifeless> huwshimi: isn't it?
<StevenK> NCommander: Tonight is the LP team dinner
<StevenK> I'd suggest this afternoon
<NCommander> StevenK: that works.
<StevenK> NCommander: Pop your head into the LP breakout room and ask wgrant and bigjools
<lifeless> huwshimi: even if it is the same one, the proposed change wouldn't alter that portlet on bugs.l.n, as its already in the same order as the applications bar
<NCommander> StevenK: I'll be over in ~30 minutes
<huwshimi> lifeless: The proposal is that all those portlets would be the same as the main tabs
<huwshimi> lifeless: I don't think it is a bug that they are in a different order
<huwshimi> (a different order to the main tabs)
<lifeless> huwshimi: I thought the reported defect was that the order was inconsistent
<lifeless> huwshimi: not that they needed the same content
<huwshimi> lifeless: yes
<huwshimi> lifeless: Are we not saying the same thing?
<lifeless> I thought for a statement there that you thought the bug was larger than the order
<lifeless> 'same as the main tabs'
<lifeless> huwshimi: so I think its fine to say 'the order is different because (reason)' and wontfix it on that basis.
<lifeless> huwshimi: closing something we agree is a defect really rubs me the wrong way; in the bug so far you have not indicated a preference for the status quo, rather indicated that change might (and we have no data only guesses) be expensive
<huwshimi> lifeless: Well the bug merely says they should be the same for "consistency"
<huwshimi> lifeless: There is no data to suggest that that is true either
<lifeless> well
<lifeless> there is some research out there, but I don't think its conclusive
<huwshimi> lifeless: Can you point me towards it?
 * lifeless digs
<lifeless> Nielsen 1989b, http://en.wikipedia.org/wiki/User_interface#Consistency has some references
<huwshimi> lifeless: Oh sorry I thought you meant something specific to this bug
<lifeless> huwshimi: we don't have data on the cost of change; we don't have data on the cost of the inconsistency - thats true
<lifeless> we have some models about potential costs for the inconsistency; and we can data mine logs to assess the cost of change if we want to
<lifeless> huwshimi: so I'm not invested either way; As I read the report its a valid defect - but I wouldn't have called it high.
<lifeless> huwshimi: my objection to closing it is based around whether its a valid defect or not; we shouldn't close bugs just because its tricky to do right now.
<huwshimi> lifeless: I don't believe it is a valid defect because there is an inconsistency. As curtis mentions on the bug the order was chosen (however arbitrary that may have been).
<huwshimi> lifeless: It is a consideration, sure, but not a bug
<lifeless> huwshimi: curtis also says 'We want one order for all three portlet'
<huwshimi> lifeless: Possibly we don
<huwshimi> *we do
<huwshimi> lifeless: It is not *wrong* that they are different
<lifeless> huwshimi: being *wrong* is not a necessary condition to be in the tracker
<huwshimi> lifeless: No, that's why we have statuses like "won't fix"
<lifeless> huwshimi: huh? thats unrelated
<lifeless> we have wontfix for things we don't want to do.
<huwshimi> lifeless: Right, and I don't think we want to fix this
<huwshimi> lifeless: At least not just for the sake of making it "consistent"
<lifeless> thats not consistent with what curtis wrote earlier in the bug
<lifeless> (heh, sorry about the reuse of consistent, brain-broke)
<lifeless> I'm concerned that we're closing off something we would like to do (have them consistent) because the cost of change is higher than we're willing to pay for consistency.
<lifeless> Thats getting trapped in a local trough
<lifeless> a better thing to do is to say that we won't change it *solely* for this, leave the bug open, and when we have other reasons to change the portlets do this at the same time.
<lifeless> the aspects of 'would we like this' and 'what is necessary to do it well/acceptably' are quite separatabl
<lifeless> e
<abentley> henninge: The RosettaApplicationView dates back to June 12 2005, and was last seriously touched by SteveA.
<lifeless> huwshimi: this is dragging on too long
<huwshimi> lifeless: So lets change this to something like opinion then?
<lifeless> huwshimi: whats your objection to it being open with a caveat about when to do it?
<lifeless> huwshimi: have I misunderstood you? do you actively want the portlets to have different orders?
<huwshimi> lifeless: It's possible that a different order is actually better
<lifeless> huwshimi: do we actually think that? curtis said that he wanted them to have the same order eventually
<lifeless> huwshimi: if you think a different order *is* better, wontfix is totally fine, but as I understand it you haven't said that.
<lifeless> huwshimi: you seem to be on the fence about that aspect, and unwilling to inflict a (possibly traumatic) change on our users for something you are ambivalent about
<lifeless>  - which is totally reasonable
<lifeless> however, curtis was totally clear that he wants all these things to have the same order, which agrees with the reporter, and combining these views suggests a real bug we shouldn't do without either more data about costs/benefits || some other change to alter the cost/benefit ratio by doing at the same time
<huwshimi> lifeless: I would be happy with a bug that said "the current order of portlet links may not be the best possible order"
<benji> mrevell: I changed 728193 to "In Progress"
<lifeless> huwshimi: so change this one to say that :)
<huwshimi> lifeless: But I don't think that's really a bug
<lifeless> I'm getting really confused here
<StevenK> jcsackett: https://bugs.launchpad.net/launchpad/+bug/365812
<_mup_> Bug #365812: lazr-sam.css should be placed under lazr/build/assets/ along with required image files <easy> <javascript> <lp-foundations> <performance> <ui> <Launchpad itself:Triaged> <LAZR Javascript Library:Triaged> < https://launchpad.net/bugs/365812 >
<lifeless> you're not stating that consistency is undesirable; you acknowledge that we may want them to be consistent; curtis *does* want them to be consistent, and the only reason presented not to harmonise things is the cost of change
<mrevell> thanks benji
<huwshimi> lifeless: That seems to be more of an opinion thing to actually being a bug
<lifeless> huwshimi: sure, but its an opinion that the guy which brought in this portlet agrees with
<lifeless> huwshimi: I've acknowledged your constraint over cost-of-change, but you still don't want this (valid) bug in the system for some reason.
<lifeless> and I don't understand that
<lifeless> if you're happy as things are, then cost of change is irrelevant,a nd closing it is appropriate
<huwshimi> lifeless: So I also think there are bigger things here that make this issue kind of redundant, one of which I mentioned on the bug (about not having the "participation" portlet)
<lifeless> huwshimi: if we've committed to doing something about it, closing wontfix (because we are doing X whichmakes redundant) also makes sense
<lifeless> huwshimi: but if we haven't committed to doing that, or if doing that might not solve this thing, then they are different issues
<huwshimi> lifeless: I don't really care how this is resolved. I don't think it's valid to say the portlet order should be the same as the nav which is why I closed as won't fix
<lifeless> ah, so this is the meat :)
<lifeless> huwshimi: do you know why curtis said that we *do* want one order for them all ?
<huwshimi> lifeless: nope
<lifeless> I suggest you talk to him, since you are both there.
<lifeless> *if* we want one order, then this bug is squarely on the issue and we should update it to make sure we don't change this willy-nilly but coordinate it at some future time.
<lifeless> if we don't want one order, then the argument about cost of change is irrelevant and wontfix is fine (but we should update it to be clear that its not wontfix-this-is-hard, its wontfix we don't *want* to be consistent.
<lifeless> What I want is that the reasoning is clear, so that the reporter understands why, which helps them understand how we're building LP better (lots of our reporters report > 1 bug IME, and we want to help folk become contributors)
<lifeless> huwshimi: I'm crashing now before midnight arrives
<huwshimi> lifeless: Ok thanks for that
<lifeless> huwshimi: thanks for your patience humouring me on this; I look forward to *one* of those things in the bug report in the morning :)
<lifeless> unclear bug closing tends to set me off (because of the social impact it has)
<lifeless> huwshimi: I realise this discussion probably seems a lot larger than you expected when you closed the bug :)
<lifeless> anyhoo
<lifeless> night all
<lifeless> have a great day sprinting
<wgrant> Night lifeless.
<matsubara> anyone available for a quick review: https://code.launchpad.net/~matsubara/launchpad/772016-mailinglist-help/+merge/66443?
<rvba> wgrant: http://paste.ubuntu.com/635746/
<wgrant> rvba: We could have routing keys like lp.job.MergeProposalJob.1234, or lp.job.MergeProposalJob.1234.status_change, or lp.job.MergeProposalJob.1234.status.completed, or...
<rvba> wgrant: that's the plan yes.
<jcsackett> sinzui: how does one fire off all YUI tests at once? something with layer=YUI... yeah?
<wgrant> jcsackett: --layer=YUI should do it.
<jcsackett> thanks wgrant.
<matsubara> jelmer, thanks for the review!
<jelmer> matsubara: anytime :)
<benji> StevenK: boo!
<benji> StevenK: will you review https://code.launchpad.net/~benji/launchpad/bug-436247/+merge/66336 for me?
<StevenK> benji: I could be convinced. Whaddya got?
 * benji hands StevenK ten thousand units of his currency of choice.
<StevenK> Haha
<StevenK> benji: My only concern is you drop a tal:define of number_of_dupes
<benji> ooh, looking
<benji> StevenK: yep, that block didn't have anything in it (and defines are scoped to the enclosing tag (unless they have "global" on them))
<StevenK> benji: Right, so it was utterly pointless
<benji> absolutely
<StevenK> benji: r=me
<benji> cool, thanks
<jml> poolie: Give lp:~jml/testtools/gassy-failure-660852 a try if you'd like
<wgrant> allenap: http://paste.ubuntu.com/634326/
<benji> StevenK: that was lazr.restfulclient, right?
<StevenK> benji: Right
<benji> k
<benji> StevenK: I don't see a NEWS.txt entry.  What's a good one-line description of the change?
<StevenK> benji: Best person to ask is poolie.
 * benji sends telepathic waves towards poolie.
<StevenK> Did they work? :-P
<benji> not yet
<benji> I think I'll just construct some NEWS entries from the bzr log, it looks like several people merged things in without updating NEWS.
<benji> Bad people.  No cookie.
<StevenK> benji: Haha, thank you for sorting it.
<huwshimi> deryck: Do you want to review this branch? https://code.launchpad.net/~huwshimi/launchpad/bug-link-focus-749845/+merge/66474
<deryck> huwshimi, sure.  5 minutes to finish another review, and I'll take a look.
<huwshimi> deryck: np
<deryck> wallyworld_, hey.  you really here?
<wallyworld_> deryck: yes
<wallyworld_> sometimes
<deryck> wallyworld_, actually hold on, sorry.  taking my advice to jcsackett for a make clean first.
<wallyworld_> ok
<deryck> wallyworld_, false alarm.  works beautifully and diff reports the same files now.
<wallyworld_> deryck: \o/
<deryck> wallyworld_, so r=me.
<wallyworld_> awesome, thanks
<deryck> np
<deryck> huwshimi, looking at your MP now.
<deryck> huwshimi, come see me about your bug when you can.
<deryck> or your MP rather
<huwshimi> deryck: Sure, be there in a few
<deryck> huwshimi, cool
<benji> StevenK: ok, 0.12.0 is released
<StevenK> benji: Thanks! Where can I grab it?
<benji> StevenK: https://launchpad.net/lazr.restfulclient/+download
<poolie> thanks jml
<jml> poolie: you might want to try again with the latest branch. I've applied mgz's suggestions.
<deryck> huwshimi, http://pastebin.ubuntu.com/635875/
<huwshimi> deryck: Thanks for that
<huwshimi> deryck: Do you want to approve the branch, or are you ok for me to do it?
<deryck> huwshimi, no problem.  I added the pastebin to the MP when I approved it just now, too.
<huwshimi> deryck: Ah awesome thatnks
<deryck> np
<huwshimi> *thanks
<huwshimi> poolie: If you want to talk about the branch page any more I'm available now
<allenap> rvba: lp:~allenap/launchpad/routing-key-generation
<gmb> wgrant: http://oo00.eu
<poolie> huwshimi, mrevell, would you two like to come and talk about bug workflow with ubuntu people? skaet etc
<poolie> up on floor 2
<huwshimi> poolie: sure, on way
<jml> deryck: re "Big Bang Theory", http://life.metagnome.net/2011/05/crane-transposition.html
 * deryck looks
<wgrant> gmb: You appear to have failed to push your lazr.amqp changes. trunk still thinks it's 0.0.2.
<timrc> wgrant, ping
<timrc> or anyone, what's the proper way to branch a private tree and push it back to LP such that it retains its privacy settings?
<dobey> timrc: if you're pushing to a project that has branches private by default, it will be private by default
<timrc> dobey, not from my experience
<timrc> dobey, at least, not with bzrlib
<dobey> bzr is not a private project
<dobey> its branches are public by default
<timrc> dobey, no, I branched a private branch, made some changes, and pushed a new branch back using bzrlib
<timrc> the new branch was public
<beuno> timrc, that decision is made by launchpad, not bzr
<beuno> so it depends on the location
<dobey> pushed to where?
<beuno> if you push back to the project with private branches by default, it will be private
<dobey> timrc: so you're writing some script that uses bzrlib, and does the pushing/pulling bits?
<timrc> dobey, yeah
<timrc> maybe it's because the private branch is owned by a user and not a project?
<beuno> it shouldn't matter
<dobey> projects don't own branches
<timrc> a team, rather
<beuno> you need to find out if a project has private branches by default or not
<lifeless> moin
<dobey> could be the project does not have private branches by default
<bryceh> While:
<bryceh>   Installing scripts.
<bryceh>   Getting distribution for 'lazr.amqp==0.1'.
<bryceh> Error: Couldn't find a distribution for 'lazr.amqp==0.1'.
<bryceh> this is from attempting a rocketfuel-branch after updating to latest devel today.  ideas?
<lifeless> update your download cache
<bryceh> yep, I think that did it
<bryceh> thanks
<timrc> I think my problem is that I pushed to +junk ?
<timrc> are branches that get pushed to +junk made public?
<lifeless> timrc: +junk is public
<timrc> so this is just my general lack of understanding :)
<lifeless> timrc: there is /no/ concept of 'is this thing private' for bzr
<lifeless> timrc: so once you have a local branch, its just like every other branch
<timrc> lifeless, this is just a result of my misunderstanding
<lifeless> timrc: /projects/ in launchpad can define privacy rules
<timrc> lifeless, okay
<dobey> hrmm
<dobey> why is most of the content in http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/maverick/gobject-introspection/maverick/view/head:/debian/control.in red text on red background? makes no sense to me :)
<lifeless> looks lik a pygments fail
<lifeless> dobey: file a bug ?
<dobey> ok
<dobey> against lp, or against... whatever that thing is called that is used for the code browsing?
<beuno> dobey, loggerhead
<lifeless> lp
<beuno> against loggerhead
<lifeless> we'll add tasks to loggerhead and a watch to pygments
<lifeless> beuno: fixed-in-loggerhead isn't fixed-in-lp, so it needs both
<beuno> true
<StevenK> lifeless: O hai. Can I trouble you for a small review?
<lifeless> of course
<StevenK> lifeless: https://code.launchpad.net/~stevenk/launchpad/branch-approximatedate/+merge/66480
<lifeless> thats reviewed :)
<StevenK> lifeless: Thanks. I didn't think it would be controversial :-)
<StevenK> lifeless: As an aside, 'a moment ago' is from the branch I landed a few hours ago
<lifeless> no worries
<lifeless> yeah, I like the look of it
<StevenK> I'm not sure if 10 seconds is the right answer, but it's only mentioned in one place ...
<lifeless> That test could fail, but if wehave 10 second swap storms stalling the test runner, we have a real problem
<StevenK> Right
<lifeless> hows the week been ?
<StevenK> Tiring
<StevenK> And in fact, on that note, I'm going to bed. :-)
 * jelmer waves
<mwhudson> jelmer: hi!  fixed that bzr-svn bug yet? :)
<jelmer> mwhudson!
<jelmer> mwhudson: No, but making progress
<mwhudson> cool
<mwhudson> nice to see some work on loggerhead, too
<jelmer> mwhudson, I did some manual imports of gcc today to unblock some people
<jelmer> mwhudson, and I at least have an idea how to manually fix the issue by disabling tags for the moment
<jelmer> mwhudson, the memory problem turns out to be the fact that we do one run of "svn log" per tag, and then keep the results of that in memory
<mwhudson> ah
<jelmer> gcc has ~700 tags
<mwhudson> i can see how that would be a problem for gcc
<jelmer> the fact that we hold that data in memory 700 times is accidental - we start iterators that we don't fully consume; but because we keep a handle to them, and because they consume data generated by independent threads, we use all that memory
<mwhudson> ah
<mwhudson> nice software abstractions defeat efficient performance
<mwhudson> or something?
<jelmer> yeah, though s/nice// because they defeat efficient performance :P
<mwhudson> s/^/seemingly /
<lifeless> how does one tell a doctest to run in a layer?
<lifeless> a DocTestSuite specifically
<mwhudson> lifeless: have you encountered test_system_documentation yet?
<lifeless> probably
<lifeless> I'll poke at that
<mwhudson> ah, actually i guess lib/lp/$app/test/test_doc.py is the modern equivalent
<mwhudson> (but test_system_documentation is still there too)
<lifeless> ah
<lifeless> suite.layer = x
<timrc> I'm getting http://pastebin.ubuntu.com/636053/ when I create (bzr init) a new tree and then push it to an existing project... any idea how I can rectify this?  Appears to be some sort of compatibility issue...
<lifeless> #bzr might be a better place to ask :)
<timrc> lifeless, good point :)
<timrc> my brain has married the two projects together :/
<jelmer> they're not married, but at least very good friends :)
#launchpad-dev 2011-07-01
<wgrant> timrc: Hi.
<wgrant> Ah, I se.
<lifeless> StevenK: I know why parallel is hanging
<lifeless> StevenK: contention on template1 copies.
<lifeless> StevenK: and on launchpad_ftest_template, I have a patch to sort that, but I'm getting the template1 (lower level tests) sorted first
<lifeless> OperationalError: source database "template1" is being accessed by other users
<lifeless> DETAIL:  There are 6 other session(s) using the database.
<lifeless> hmm, 30 seconds to do a full schema setup including buildout wadl etc
<lifeless> plausible to make it inline on layer setup
<wgrant> lifeless: scan_branches seems to be hung.
<wgrant> Can you look at it, please?
<lifeless> oh joy
<lifeless> thanks
<LPCIBot> Project devel build #851: FAILURE in 5 hr 0 min: https://lpci.wedontsleep.org/job/devel/851/
<james_w> lifeless, hi, how do I get subunit-filter or something to not print the "time:" lines?
<lifeless> james_w: hi
<lifeless> james_w: the time: markers will be consumed by whatever human UI you use
<lifeless> james_w: e.g. subunit2junixml, subunit2pyunit, tribunal, subunit-stats
<james_w> my eyes don't consume it :-)
<lifeless> if you're seeing hundreds of adjacent time: statements, get a new python-subunit, it now coalesces them
<james_w> I was using subunit-filter to just get me the skipped test so I could see what it was
<lifeless> yeah
<james_w> adding subunit2pyunit helped, thanks
<lifeless> thats the case where you used to see many time statements together
<lifeless> subunit-ls may be useful for you too
<lifeless> anyhow, that case is fixed in newer subunits
<james_w> subunit-ls: AttributeError: 'NoneType' object has no attribute 'id'
<james_w> so not really :-)
<lifeless> as is that :)
<james_w> yeah
<james_w> 0.0.7 works much better, thanks
<lifeless> wheee
<lifeless>  306 /   36  Distribution:EntryResource:searchTasks
<lifeless>      154 /   37  Distribution:+bugs
<lifeless> need to do someting about that
<lifeless> well
<lifeless> the world must be ending - ms shipping gcc
<lifeless> wow, postgresql, creating a db isn't really tuned is it
<mwhudson> lifeless: someone did an experiment years ago where lp untarred a db rather than creating one with pg; didn't save any time
<lifeless> mwhudson: bin/test --parallel -vvt canonical.database.postgresql
<lifeless> you'll want the bugfix from my paralleltests branch
<lifeless> or they will try to create the *same* test db
<lifeless> mwhudson: we're seeing 10 second delays sometimes
<lifeless> with multiple threads making db's
<mwhudson> lifeless: ah so it's kinda overdone locking or something?
<lifeless> underdone it seems
<lifeless> createdb works by cloning (e.g. template1)
<lifeless> if two users try to clone at once, the *both* start, and then *both* fail ('other users are accessing template 1')
<mwhudson> oh
<mwhudson> nice
<lifeless> s/the/they/
<lifeless> 200ms to clone template1 on my new machine
<lifeless> 8 hardware threads
<lifeless> how often do you think I hit this contention ?
<lifeless> being able to set a connection limit on the db helps
<lifeless> but it varies wildly
<lifeless> even with a bzr lock around our create db calls:
<lifeless> 1309494289.41 (12.18) completed (1 retries) 4796 template1
<lifeless> released 4796
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #852: FIXED in 5 hr 7 min: https://lpci.wedontsleep.org/job/devel/852/
<lifeless> wgrant: did you end up getting lxc rabbit fixture happy?
<wgrant> lifeless: No. Need to investigate that on mondayish.
<lifeless> I have bad news about non-lxc parallel tests
<lifeless> 311 seconds (serial) -> 92 (parallel) with my 8-core, and I think its all contention on the db reset. I need to profile more.
<lifeless> bzr+ssh://bazaar.launchpad.net/~lifeless/launchpad/paralleltests has my stuff so far
<wgrant> lifeless: Is that with multiple templates?
<wgrant> lifeless: Have you tried multiple clusters?
<lifeless> wgrant: thats with a double-template
<lifeless> wgrant: each runner forks l_f_t to l_f_t_$pid
<lifeless> wgrant: then makes teset instances from the l-f_t_$pid
<lifeless> wgrant: I'm going to do family time now, but perhaps you'd like to review https://code.launchpad.net/~lifeless/launchpad/paralleltests/+merge/66554
<wgrant> rvba: You'll need to add 'job.notifications-queue.' to the subscribe_key when binding.
<wgrant> Well, 'job' is at your discretion.
<wgrant> allenap: too ^^
<nigelb> gmb: Great picture of the community team!
<gmb> Ta
<nigelb> dholbach just told me you did the magic there :)
<jml> At the lightning talk, forgot to say "Trust pyflakes"
<jml> 'bzr ls -VR --kind=file --null | xargs -0 grep -In %s' is also a good replacement for 'bzr grep'
<wgrant> flacoste: sudo -u archvsync /home/archvsync/scripts/schedule-logsync.sh staging
<Riddell> StevenK: do you have the group photo?  I'd like to do a blog post
<Riddell> or gmb?
<bac> Riddell: it is on the wiki, somewhere
<bac> Riddell: https://wiki.canonical.com/Launchpad/Sprints/Thunderdome2011
<jtv> danilos, spiv: expander landing
<jml> hey
<jml> there's some scripts out there to automate the Launchpad part of releasing projects
<jml> does anyone remember where they are?
<jml> poolie: ^
<spiv> jtv: yay!
<jml> lp:lptools, for those burning with curiosity
<jtv> Does reading a feature flag from TAL (tal:condition="features/â¦") require the page to be on a LaunchpadView?
<jtv> I'm getting KeyError: "features"
<spiv> request/features perhaps?
<jml> any way to automate uploading a tarball & sig for a release?
<jelmer> jml: ubuntu-dev-tools has a script for uploading a tarball IIRC
<jml> jelmer: ta
<jelmer> jml: lp-project-upload
<lifeless> also there is the kraken
<lifeless> and stuff in lptools
<jtv> spiv: no, it's "features/" not "request/features/" â plenty of other templates use it.
<jtv> It's not the LaunchpadView though.
<jtv> AFAICT this does use one.
<lifeless> its mapped as a macro in the base template
<lifeless> that expands features to request/features I believe.
<jml> lifeless: lptools hides its ability to upload stuff
<lifeless> night all, have a good flight back your various homes
<jml> I've never heard of the kraken in context of launchpad
<jtv> good night lifeless!
<jml> lifeless: g'night.
<spiv> jtv: grep suggests plenty of other templates use request/features too :)
<lifeless> jml: ask abentley about it
<jtv> ah :)
<benji> huwshimi: https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html
<jtv> spiv: https://code.launchpad.net/~jtv/launchpad/bmp-inline-diffs/+merge/66591
<jtv> spiv: the feature we're working on would actually have been helpful in producing that branch!
<thumper> abentley: are you near sinzui?
<abentley> thumper: No, I don't know where he is.  I'm in the downstairs room.
<thumper> abentley: ok, thanks
<abentley> thumper: Oh, I see him now.
<thumper> abentley: could you tell him I want to talk to him about launchpad on oneiric
<abentley> thumper: where are you?
<thumper> abentley: DX breakout 2nd floor
<abentley> thumper: he's on his way.
<thumper> abentley: ta
<adeuring> deryck: https://code.launchpad.net/~launchpad-orange-squad/launchpad/bug-740208-obfuscate-ws/+merge/66583
<StevenK> abentley: https://code.launchpad.net/~stevenk/launchpad/approximateduration-no-words/+merge/66586
<abentley> StevenK: A dict indexed by integer makes my head hurt.
<StevenK> Yeah .... It isn't very plesant code
<wgrant> rvba: ch.queue_declare('bdirect', arguments={"x-expires": 600})
<wgrant> So, x-expires: 60000 or so.
<bigjools> LONG POLL SQUAD HAS MADE FIRE
 * benji wonders if bigjools was slightly confused about his goal.
<bigjools> you just need to think of Tom Hanks on the beach of his island getaway
<StevenK> allenap: https://code.launchpad.net/~stevenk/launchpad/branch-subscriber-vertical-space/+merge/66615
<poolie> woo
<jtv> woo?
<jtv> woo who?
<jtv> â¦or whom?
<nigelb> So, no team photo for launchpad team? :D
<poolie> there is one somewhere
<poolie> woo FIRE
<Riddell> nigelb: see planet.kde
<nigelb> Riddell: hah
<nigelb> Riddell: tucked away on another planet :P
<Riddell> jcsackett: is 421664-code-tab-hover-text forever fated to fail?!
<bigjools> allenap: http://paste.ubuntu.com/636395/
<nigelb> is bigjools the tall guy on the right?
<bigjools> never seen 'im in me life guv
<nigelb> heh
<nigelb> I can recognize only 6 faces :(
<jcsackett> Riddell: I'm going to run those failing tests locally; they don't look like anything your branch introduced.
<bac> abentley: my branch is at lp:~bac/launchpad/getnewcache
<abentley> bac: cool, thanks!
<jcsackett> Riddell: wasn't able to directly land your branch, so its out through the normal process again.
 * jcsackett crosses fingers.
<StevenK> spiv: Can haz bug about old FF removal?
<StevenK> spiv: It should be fairly easy to rip it out -- and I'll be bored on the plane ...
<spiv> poolie: http://people.canonical.com/~andrew/Inline-diff-screenshot.png
<poolie> ta
<wallyworld> wgrant: meet at 7:30 tomorrow for breakfast?
<wgrant> wallyworld: Sure!
 * wgrant sleeps.
#launchpad-dev 2011-07-03
<LPCIBot> Project db-devel build #691: FAILURE in 5 hr 43 min: https://lpci.wedontsleep.org/job/db-devel/691/
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #692: FIXED in 5 hr 40 min: https://lpci.wedontsleep.org/job/db-devel/692/
<wgrant> Morning.
<lifeless> wgrant: olÃ¡
<wgrant> Barely.
<lifeless> we have a 150cm climbing tree for the cats
<lifeless> they are both wrestling at the top of it right now
<wgrant> :(
<lifeless> and now are sleep on each other
<lifeless> wgrant: are you planning on working today or just going 'blargh'
<wgrant> lifeless: Going to try to get us deployable, then blargh.
<lifeless> wgrant: if the former, I heard a rumour you'd made progress on rabbit
<wgrant> And maybe try to get rabbit sorted out.
<wgrant> Since that is mostly poking spm and considering packaging rabbitmq-management.
<lifeless> rabbitmq-management ?
<wgrant> A plugin which provides an HTTP API which will allow us to reset the fixture in a couple of ms.
<lifeless> cool
<wgrant> Natty has a plugin or two packaged, so it should be easy enough.
<lifeless> did I tell you that db resets are 0.2s
<wgrant> gary says they were taking 4s for him (when using the JS factory/teardown bindings written last week).
<wgrant> I need to investigate that this week.
<wgrant> Since it sounds implausible.
<wgrant> By a factor of 10 or so.
<wgrant> lifeless: Did you see the stub/bigjools Messaging API v3 land?
<lifeless> kindof
<lifeless> wgrant: what was taking 4s?
<wgrant> lifeless: I'm not entirely clear on that. But it was one of the normal layer teardowns, AIUI.
<wgrant> It only came up during the wrap-up.
<lifeless> so
<lifeless> if there is any contention on the template db
<lifeless> both copies will fail
<wgrant> I suspect there was an appserver still running or something.
<lifeless> and they take their own time about it
<wgrant> Since it's in the YUITestLayer-with-appserver monstrosity.
<lifeless> can I get a review of https://code.launchpad.net/~lifeless/launchpad/paralleltests/+merge/66554 ?
<StevenK> No, since the diff hasn't generated yet. :-P
<lifeless> it won't
<lifeless> the scanner blew up
<lifeless> bzr diff -rr ancestor:lp:launchpad https://code.launchpad.net/~lifeless/launchpad/paralleltests
<lifeless> should get you a diff
<StevenK> Then pastebin it? :-)
<lifeless> sure
<lifeless> I've linked a pastebin into it
<StevenK> I reserve the right to sob and cry and state it's too hard to review while jetlagged.
<lifeless> of course
<lifeless> this is just test plumbing
<lifeless> it should fix the parallel test job hanging
<StevenK> To my jetlagged adled brain, it looks fine.
<StevenK> I approve of more logging.
<lifeless> the logging is crude, but it helped me a lot when fiddling with it, and I expect others will need said fiddling themselves
<lifeless> e.g. future me
<StevenK> lifeless: As an aside, I'm curious how the scanner blew up on the branch?
<lifeless> StevenK: bug 612171
<_mup_> Bug #612171: Error generating merge proposal diff when "source branch has pending writes" <code-review> <lp-code> <Launchpad itself:Triaged> < https://launchpad.net/bugs/612171 >
<StevenK> lifeless: I've approved the branch, land it, ping me when it has and I'll re-enable the Jenkins job.
<lifeless> I hit that
<lifeless> (I'm using the term scanner loosely)
<StevenK> I wonder if we could fix that using Julian's work to suspend the job
<lifeless> you could
<StevenK> If .checkReady() returns False, suspend the job for one run and unsuspend it on the next run.
<lifeless> I think its simpler to just cancel the job cleanly
<lifeless> and when scanning a branch, we should be updating the merge proposal(s) for it anyway
<StevenK> It will or will not get scanned again by itself?
<lifeless> right
<StevenK> That was a question, not a statement. :-P
<lifeless> 'write pending' is a state that will either rectify itself (and thus the rectification can create new merge proposal jobs), or it won't (and thus a suspended job would never succeed)
<StevenK> Right. Then I agree with you.
<lifeless> polling every run is inefficient and can fail horribly (when the things being polled but not active grows large)
<StevenK> We should kill the job
<lifeless> I suspect we have a hole in our 'create a new mp job' logic at the moment, the flip side of this error
<lifeless> another option would be to ignore the pending write, and read the *old* tip
<lifeless> anyhoo
#launchpad-dev 2012-06-25
<StevenK> wallyworld_: Can you QA r15475 soonish?
 * StevenK stabs Python process handling.
<wgrant> StevenK: What are you doing and why?
<StevenK> wgrant: Trying to debug AuditorFixture._stop()
<wgrant> Ah, that.
<lifeless> StevenK: SIGPIPE again ? :>
<StevenK> lifeless: Already SIG_IGN'ing PIPE
<StevenK> It drops out of _stop() without actually killing the process.
<cjwatson> I've QAed r15480; it has problems which mean the new API isn't usable, which I'll work on tomorrow, but as far as I've been able to determine there are no regressions in existing functionality, so I marked it qa-ok.  Feel free to check my reasoning in bug 334858.
<_mup_> Bug #334858: Require a way to copy [P]PPA packages into Ubuntu <lp-soyuz> <qa-ok> <Launchpad itself:In Progress by cjwatson> < https://launchpad.net/bugs/334858 >
<wgrant> cjwatson: Right, sounds good. That usually means a missing commit, indeed.
<wgrant> Since you can't read back an LFA's content unless it's been committed.
<cjwatson> Too much to hope that I'd get it all right in one go.  I wonder how delayed-copy processing manages to get it right, since I can't see a commit there.  Maybe it's implicit somewhere.
<wgrant> Delayed copies have regressed in that respect once or twice
<wgrant> I forget how it works
<cjwatson> Maybe it works because delayed copies get hold of the changelog manually rather than going through the SPR.
<cjwatson> As in, from the .changes, which PCJs can't do.
<wgrant> Ah, yes, that would be it.
<wgrant> But some aspect of delayed copies sometimes tried to read it back
<wgrant> And died
<wgrant> Possibly rejection
<wgrant> Can't quite remember
<wgrant> Anyway, thanks for QAing
<cjwatson> Clearly means I need to add a test for the PCJ as well as the PackageCopier path, anyway.
<cjwatson> Grumble.
<cjwatson> I suspect also I need to do pocket queue admin permissions before we can kill unembargoing and the ubuntu-security celebrity, since doing it with PCJs means their uploads will all wind up in unapproved.
<wgrant> Indeed. That's going to be a little awkard on +queue, I suspect, but shouldn't be too much effort.
<wgrant> Hm, actually.
<wgrant> How's that going to work at all
<wgrant> Because they'll still be private..
<cjwatson> In the queue it'll just be a suspended PCJ
<wgrant> Yeah
<wgrant> Hopefully it won't break when it tries to access its attributes.
<cjwatson> Oh, I hope it doesn't need to ... yeah, that
<cjwatson> I think it's OK.  The attributes all come from the PCJ's metadata dict or similar.
<wgrant> Great.
<wallyworld_> StevenK: sorry, was at locksmith, qa done now
<StevenK> wallyworld_: More mailbox trouble? :-)
<wallyworld_> StevenK: yes, bloody keys i got didn;t fit, i had to break in and rip off the front flap and tale it all in
<wallyworld_> the lock turned out to be a cheap chinese version and there are no key blanks here
<StevenK> wallyworld_: Ah, so you were so desperate for the phone cover that you broke into the mailbox. :-)
<wallyworld_> StevenK: that arrived last week and i had to use a coathander to hook the package and lift it out
<StevenK> Now that is desperation.
<wallyworld_> yep
<StevenK> lifeless: This is incredibly frustrating. :-(
<StevenK> Sigh. self.process.wait() returns immediately, but the process isn't dead.
<lifeless> StevenK: sorry, context switching
<lifeless> so whats up ?
<StevenK> lifeless: fixture._stop() is supposed to kill bin/auditor-manage.
<lifeless> does auditor-manage fork or something ?
<StevenK> I doubt it
<StevenK> It's the usual 'python manage.py runserver' which is used for development, so I'd be very surprised if it does.
<lifeless> ok
<lifeless> so the code looks reasonable
<lifeless> server.poll() is None implies has not exited.
<lifeless> erm
<lifeless> process.poll()
<lifeless> StevenK: what do you see happen ?
<lifeless> StevenK: have you tried using pdb and stepping through it ?
<StevenK> lifeless: I've tried printf debugging, which hasn't been much help.
<lifeless> do you want video + debug help?
<lifeless> right now I don't know what you've learnt.
<lifeless> so I'm starting cold.
<StevenK> lifeless: Today I've learnt that 1) I either don't understand process workflow under Unix or this thing is misbehaving, 2) I feel like I've wasted today on it. :-)
<lifeless> the day isn't over.
<lifeless> lets free our hands for actual examination: skype or g+ ?
<StevenK> lifeless: Would you mind looking sans Skype/G+ for 10-15? I've just taken Advil for a headache.
<lifeless> not at all
<lifeless> so, tell me a) how to reproduce and b) all that you have learnt.
<lifeless> is your code up online etc.
<StevenK> lifeless: Just pushed auditor
<lifeless> ?win 85
<StevenK> lifeless: auditorfixture updated
<StevenK> lifeless: Easiest way to reproduce is run the testsuite. One test will fail, but if you look at ps aux afterwards you'll see two auditor-manage's hanging around.
<StevenK> lifeless: What I have learnt is that the _stop() method does get called, and it does call self.process.terminate(), and the next time around the loop self.is_process_up returns False and it exists, but the process is still running.
<StevenK> s/exists/exits/
<lifeless> ok
<lifeless> so lets get some more details
<lifeless> StevenK: so, I'm putting a breakpoint in _stop. And running the test suite
<lifeless> StevenK: so it gets terminated via SIGTERM
<StevenK> lifeless: Are the processes still running, though?
<lifeless> StevenK: but nothing is reaping it
<lifeless> so it sits defunct
<StevenK> But Python should catch the SIGCHLD and reap it?
<lifeless> no
<lifeless> not AFAIK
<lifeless> also
<lifeless> you may well be running into a signal handling race condition
<lifeless> lets see
<StevenK> I thought the entire point of subprocess was abstract out this mess. :-P
<lifeless> but let me check the code a bit more
<lifeless> StevenK: so, its kill the process, reliably.
<lifeless> StevenK: compare:
<lifeless>  :!bin/test
<lifeless> Running zope.testrunner.layer.UnitTests tests:
<lifeless>   Set up zope.testrunner.layer.UnitTests in 0.000 seconds.
<lifeless> 23890
<lifeless> thats the pid
<lifeless> then
<lifeless> 23890
<lifeless> 23894
<lifeless> 23894
<lifeless> (twice through apparently)
<lifeless> but
<lifeless> compare the leaked processes:
<lifeless> robertc  23895  5.7  0.2 206692 19168 pts/10   Sl+  17:05   0:00 /usr/bin/python /home/robertc/source/launchpad/auditorfixture/working/auditorfixture/tests/../../bin/auditor-manage runserver 9999
<lifeless> robertc  23891  5.7  0.2 206692 19164 pts/10   Sl+  17:05   0:00 /usr/bin/python /home/robertc/source/launchpad/auditorfixture/working/auditorfixture/tests/../../bin/auditor-manage runserver 51284
<lifeless> note the pids are one out
<StevenK> Which means it probably did fork
<lifeless> robertc  23918 17.6  0.1  61964 15036 pts/10   S+   17:06   0:01                      \_ /usr/bin/python -S bin/test
<lifeless> robertc  23919  1.6  0.1  52288 15740 pts/10   S+   17:06   0:00                          \_ /usr/bin/python -S /home/robertc/source/launchpad/auditorfixture/working/auditorfixture/tests/../../bin/auditor-manage runserver 47997
<lifeless> robertc  23920  2.8  0.2 206688 19156 pts/10   Sl+  17:06   0:00                              \_ /usr/bin/python /home/robertc/source/launchpad/auditorfixture/working/auditorfixture/tests/../../bin/auditor-manage runserver 47997
<lifeless> ^ you need to be way more paranoid.
<StevenK> It double fork()'s? :-(
<lifeless> paranoid about assuming I think I mean
<lifeless> anyhow, yes, it looks like it does.
<lifeless> well, something does.
<StevenK> Single fork() should be dealt with fine because it will be the same process group, but a double fork is not, from memory.
<lifeless> if we signal the process gorup
<lifeless> by (IIRC) doing -pid
<lifeless> nah
<lifeless> the pgroup they are in is the teset servers group
<lifeless> because we're not making one when we spawn
<lifeless> should be easy to fix, one sec
<StevenK> lifeless: I really don't want to call os.setsid() in the fixture ...
<StevenK> lifeless: It wasn't so easy?
<lifeless> StevenK: lp:~lifeless/auditorfixture/pgrphandling
<lifeless> StevenK: it was, Lynne got home
<StevenK> Ah, heh.
 * StevenK looks.
<StevenK> lifeless: Bah, and you do end up using os.setsid() :-)
<lifeless> of course
<lifeless> but, it works
<lifeless> and doesn't leak defunct processes like a sieve
<StevenK> Which was the entire point of the exercise
<lifeless> it was leaking defunct *and* live ones before
<lifeless> this is twice as much betterness ;)
<lifeless> StevenK: HTH, -> dinnery stuff, will ping later to see how you went
<StevenK> lifeless: Aye
<Brooklyn> Hello, I have deployed launchpad. But I can't register new user now. Could anyone give me some hints?
<StevenK> lifeless: HAH. Now I have to put django into download-cache
<wgrant> Brooklyn: The testopenid provider that Launchpad dev instances use doesn't support registration. You'll need to point it at a proper OpenID provider.
<wgrant> lifeless: Bah
<Brooklyn> wgrant: Thanks for your reply.
<wgrant> lifeless: So it turns out that bugsummary queries for source packages are a hundred or so times slower than they need to be.
<wgrant> lifeless: There's no usable index starting with (distribution, spn).
<wgrant> lifeless: So eg. the stats portlet falls back to (distribution, status) then filter on that
<wgrant> etc
<lifeless> cool
<wgrant> lifeless: Was this a deliberate decision, do you recall?
<lifeless> no, oversight
<wgrant> Great, thanks.
<lifeless> it was intended to index all the use cases to be fast
<wgrant> Just wanted to check you didn't find some horrid postgres terribleness
<lifeless> not that I recall
<wgrant> Now the rebuild is a little faster :)
<czajkowski> morning
<czajkowski> wgrant: is it possible to properly remove a ppa instead of seeing a message ppa is deleted?
<wgrant> Not at present
<czajkowski> is it possible to re enable it once it's deleted?
<wgrant> Um
<wgrant> Sort of
<wgrant> If they go to +edit and click the Enabled and Publish checkboxes it will mostly work
<czajkowski> ah ok
<wgrant> But it's best to tell the users to build a time machine and not delete things that they don't want deleted.
<czajkowski> well this is true :)
<czajkowski> but at least I can now asnwer people who have yet to build a tardis :)
<czajkowski> wgrant: cheers
<lifeless> could someone lp-land https://code.launchpad.net/~lifeless/launchpad/sourcecode-meta/+merge/111527 for me? _still_ haven't sorted my landing story. I know, I suck.
<jelmer> hi lifeless
<jelmer> lifeless: lp-land rather than ec2 land?
<lifeless> have a look at the branch, and make your own mind up :)
<jelmer> heh, okay.. still waiting for the diff to come up :)
<wgrant> It won't
<wgrant> It timed out days ago
 * jelmer grumbles
<jelmer> lifeless: anyway, I'll land once I manage to extract a diff
<lifeless> jelmer: :>
<lifeless> jelmer: thanks, appreciated!
<Brooklyn> wgrant: I am not familiar with zope. Is there any document about how to point testopenid to an openid provider?
<wgrant> Brooklyn: This is all LP-specific stuff, not Zope. It's easiest to set vhost.openid.hostname in the launchpad-lazr.conf of the config that you're using, then change launchpad.openid_provider_vhost from 'testopenid' to 'openid'
<Brooklyn> wgrant: Thank you so much :) I will try that
<cjwatson> bac,wgrant: Could you have a quick look over https://code.launchpad.net/~cjwatson/launchpad/pcj-reupload-fix/+merge/111804 for me?  Should fix last night's QA failure.
<wgrant> cjwatson: Looking
<wgrant> I think the bac is a lie.
<cjwatson> I thought so too, but you never know
<wgrant> cjwatson: Hmm
<czajkowski> https://dev.launchpad.net/ReviewerSchedule
<wgrant> cjwatson: I wonder if we can detect if we're in a webapp request and die violently there.
<wgrant> since committing the in middle of a webapp transaction is a less than good idea.
<cjwatson> Mm, I did wonder what the rules for that were
<wgrant> "don't"
<cjwatson> I suppose it might also be possible to extract the changes from the restricted file and pass those into notify
<wgrant> The librarian sometimes makes it challenging, unfortunately.
<wgrant> Right
<wgrant> Or you could commit after the copy and before the notify
<cjwatson> There's an argument that that might be safer
<wgrant> But in the caller.
<wgrant> I forget where that is.
<wgrant> Tryue.
<cjwatson> It's only one level up, in do_copy.
<Brooklyn> wgrant: I use login.ubuntu.com and finally it works!! Thank you so much!
<wgrant> Brooklyn: Excellent!
<cjwatson> I didn't think moving it up there would be desperately useful, and it would make it harder to tell whether it's necessary due to unembargoing.
<wgrant> It's awful either way.
<cjwatson> Extracting the changes content isn't exactly a one-liner right now.
<wgrant> Awful but possibly necessary :)
<cjwatson> Though we don't need changesfile_content in this case, which helps.
<cjwatson> wgrant: Wait, isn't notification mail queued until the transaction is committed?  I could notify first and *then* update privacy.
<wgrant> cjwatson: Sneaky and mildly evil, but probably quite effective.
<cjwatson> That would still involve a bit of rearrangement but should be way easier.
<cjwatson> wgrant: Have another look?  I think that should be better now ...
<wgrant> cjwatson: Looks good, thanks.
<bac> matsubara-lunch: ping me when you get back, please.
<czajkowski> bac: good morning :)
<bac> hi czajkowski
<lifeless> night
<matsubara> bac, hi
<bac> hi matsubara.  did you want to talk at 1230UTC or 1430?
<matsubara> bac, hey, just replied to your email. let's do 1430 as I have the team stand up in 40min
<bac> matsubara: works for me.  thanks!
<matsubara> bac, de rien
<StevenK> jelmer, lifeless: I think the change you did/landed broke buildbot.
<StevenK> jelmer: I think http://pastebin.ubuntu.com/1058978/ will fix it, can you investigate?
<cjwatson> It broke EC2 too.
<cjwatson> Well, ec2 land.
<StevenK> Yeah, utilities/update-sourcecode does not like blank lines in sourcedeps.conf
<StevenK> Which lifeless added
<cjwatson> Maybe http://paste.ubuntu.com/1058982/ to preserve LoC; just as clear I think
<StevenK> cjwatson: But that doesn't work either
<cjwatson> Er, that's precisely equivalent to your patch
<StevenK> if line == '\n' or ...: does
<StevenK> cjwatson: Yes, the patch was untested at that point. :-)
<StevenK> make worked fine, it took a little longer to realise this step was before it.
<cjwatson> Ah yes.  I usually use "line = line.rstrip('\n')" at the start of loops for that kind of thing.
<StevenK> It's an ugly hack, but let me land it.
<StevenK> It's clear that sourcedeps is deprecated and lifeless has an eye to remove the whole kit and caboddle at a later date.
<wgrant> if not line.strip(): continue
<StevenK> wgrant: Does not deal with comments.
<cjwatson> Bit surprised that failure to strip \n doesn't cause other problems.
<StevenK> cjwatson: The function calls yield line.strip() after that
<cjwatson> No, it calls yield line.split()
<cjwatson> >>> "foo\tbar\n".split("\t")
<cjwatson> ['foo', 'bar\n']
<StevenK> Ah, hmmm
<cjwatson> Oh, but split() with no args is magical
<cjwatson> >>> "foo\tbar\n".split()
<cjwatson> ['foo', 'bar']
<cjwatson> Fair enough
<StevenK> I guess it counts \n as whitespace
<StevenK> lifeless: Your failure to test that comment change in sourcedeps.conf means you owe me a beer. :-P
<StevenK> cjwatson: Fix landed, ec2 should now be happy again.
<cjwatson> Yay
<jelmer> StevenK, wgrant: Sorry, we had just headed out to lunch here :(
<jelmer> Thanks for picking up the pieces
<rick_h_> wallyworld_: thanks for the reply
<wallyworld_> rick_h_: np. did it make sense?
<wallyworld_> thanks for taking a look also
<rick_h_> yea, I remember that issue and didn't think about it while reading over
<wallyworld_> rick_h_: once we have all the js out of the tal and everything in the combo loader done, it won't be an issue i don't think
<rick_h_> I need to blank out some time to go over the picker stuff at some point. You'd think between your stuff and jcsackett's stuff I'd 'get' it by now
<wallyworld_> rick_h_: that stuff is "interesting" to say the least. i'd love to know the thought process behind how it was originally done
<rick_h_> wallyworld_: yea, was going through the YUI3 cookbook and the idea of YUI blocks loading additional modules lazy-loaded seems interesting for stuff like this
<rick_h_> I wonder if some sort of master "UI builder" would help us combine TAL blocks and keep things in one use() block without loading all our JS at once
<wallyworld_> not sure. now that we have the core stuff all done, we can tackle these edge cases
<rick_h_> yea
<wallyworld_> we still gotta make dev use raw instead of min though
<rick_h_> wallyworld_: ah, yea you know what, I'll try to get something in for that today if I can. I might have some slack time for that
<wallyworld_> rick_h_: it's also been on my todo list but too litte spare time :-(
<wallyworld_> rick_h_: something i just discovered - if i have an widget attribute eg my_form: {value: Y.Node.create('<form/>')}, the attr is only inited once and all new Y.mywidget() instances have the same attr value
<wallyworld_> i have to construct the attribute vale in the initialiser
<rick_h_> wallyworld_: right, you need to use a valueFn
<rick_h_> it's like using a mutable default for a method in python
<wallyworld_> ah of course. i forgot that. i've used a valuefn previously. thanks
<rick_h_> np
<rick_h_> yea, been bit myself with that sometimes. You want to do it, but makes sense as it's only created once on JS parse stage
<wallyworld_> yeah. i just plain forgot. my brain is too full with everything
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: benji | Firefighting: - | Critical bugs: 3.47*10^2
<cjwatson> bac: Are you still looking at https://code.launchpad.net/~cjwatson/launchpad/copy-custom-uploads/+merge/111653 ?  It looks like you claimed a review there a few days ago
<bac> cjwatson: i did claim it but then signaled my intent to give it up by adding the review team back
<bac> cjwatson:  i guess i should have voted abstain
<cjwatson> Ah, I missed that subtlety
<bac> cjwatson: abstained now.  sorry i was able to get to it.
<cjwatson> No problem.
<cjwatson> benji: ^- Do you think you might be able to pick up that one?  I'm trying to make gradual progress on a sort of semi-overlapping chain of stuff, and that one would be nice to get out of the way both to unblock other things and on its own merits.
<cjwatson> I know it's a pretty soyuz-heavy branch
<benji> cjwatson: sure
<benji> cjwatson: your branch looks good
<cjwatson> benji: Thanks!  If I may have Status: Approved then I can land that
<cjwatson> And hopefully delete a swathe of ugly documentation
<benji> cjwatson: you can (and IMO) should do that yourself as it is the signal that everything about the MP that needs to be approved has been and only you know that.  I approved the code, but there may be UI elements or other things that need review (like back in the day when we had seperate JavaScript reviews).
<cjwatson> benji: I can't, because I'm not in launchpad-reviewers.
<benji> cjwatson: then I suck ;)
<cjwatson> I understand the model, but I break it by being !~launchpad-reviewers but having PQM access. :-)
<benji> cjwatson: done
<cjwatson> I believe lifeless thinks it ought to be possible to add non-Launchpadders to that team ...
<cjwatson> So yeah, right now I just have to ask, sorry.  Thanks
<cjwatson> (Maybe I mean ~canonical-launchpad-committers)
 * cjwatson finds the mail about that and goes to reply, to see if he can sort this out more permanently
<bac> hi matsubara.  is now good?  if so i'll create a hangout
<matsubara> bac, yep. let me grab my headset and something to drink
<matsubara> bac, ready whenever you are. just send me the link please.
<bac> matsubara: sent you an invite
<replaceafill> sinzui, i have a couple of questions about what's left in LP: #210821
<_mup_> Bug #210821: bug tracker list shows inactive projects <404> <bugwatch> <lp-bugs> <oops> <qa-ok> <trivial> <Launchpad itself:In Progress by replaceafill> < https://launchpad.net/bugs/210821 >
<replaceafill> sinzui, actually, the duplicated one (which i didn't notice before) LP: #857888
<_mup_> Bug #857888: Deactivated projects shown on BugTracker:+index <404> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/857888 >
<replaceafill> sinzui, here's a diff with the fix: http://pastebin.ubuntu.com/1059175/
<sinzui> I recall it
<replaceafill> sinzui, my question is that the current code uses projects + products
<replaceafill> while the IBugTrackerSet.getPillarsForBugtrackers returns products + projects
<sinzui> hmm
<replaceafill> what would be best? to change the current portlet code
<sinzui> We prefer to return small to large, so product, project-group, distro
<replaceafill> ah
<replaceafill> so i could change the porlet to return that order
<replaceafill> and getPillarsForBugtrackers already does it
<replaceafill> sinzui, my second question is, how do you test that in a functional test?
<sinzui> +1 for the code fix
<replaceafill> i mean, how do you test the order in the test
<replaceafill> is there any way to get the contents of the testbrowser and say "get me all links in this div"?
<sinzui> replaceafill, We want a new testcase  in lp/bugs/browser/tests/test_bugtracker.py that tests the view's attr.
<replaceafill> ah ok
<replaceafill> i thought i needed to test the returned html :)
<sinzui> We can either reproduce a similar setup of your previous test...
<replaceafill> sinzui, ah ok
<replaceafill> sinzui, i'll work on that and show it to you later
<sinzui> or we mock out getPillarsForBugtrackers() in the test and verify it was called
<replaceafill> sinzui, i'll go with copying my previous setup
<sinzui> replaceafill, make me the reviewer when you are ready since I already understand why the first branch did not fix the UI situation
<replaceafill> sinzui, ah ok, will do
<sinzui> replaceafill, this is a sketch of the test I would have written to verify the view differs to the the tracker set: http://pastebin.ubuntu.com/1059206/
<replaceafill> ah, mocking :)
<replaceafill> cool, i'll use that then
<sinzui> I prefer this kind of test because changes to the set's method means I need to update the set's test and the views test
<replaceafill> ah, got it
<cjwatson> benji: Oops, I have a test failure in copy-custom-uploads.  Does http://paste.ubuntu.com/1059344/ look OK to you on top of that (that's in [initializedistroseries])?  That fixes the test here.
 * benji looks.
<benji> cjwatson: yeah, looks fine; DB security has tripped me up on a couple of occasions too
<cjwatson> Right, thanks.  I think I'll kill the current test and send this through.
<rick_h_> gary_poster: do you know off the top of your head what gtk-items are needed for the python-html5-browser for JS tests?
<sinzui> rick_h_, the standard gtk setup
<rick_h_> gary_poster: sorry, I nmean in LXC that is
<rick_h_> sinzui: ok, was curious of the yellow guys had a shortlist of deps for that since the tests are running in lxc as well
<gary_poster> rick_h_, oh.  I think this would be a sinzui question. the lxc aspect should basically mean its like trying to make it work in Ubuntu server
<gary_poster> rick_h_, well...we add the launchpad-deps
<rick_h_> ok, yea I've got those, but getting an error "RuntimeError: Gtk couldn't be initializ"
<rick_h_> for the js tests from the container so assumed the normal setup is expecting some basic gtk packages to be there that steting up an lxc container leaves out
<gary_poster> which ends up adding what we need there.  Like I said, there's nothing specific to the LXC story I can think of there.  Ah rick_h_ I might have a cure for what ais you.
<gary_poster> ails, even
<gary_poster> rick_h_, first, you are using xvfb-run, right?
<rick_h_> well starting out just by doing:  ./bin/test -x -cvv --layer=YUITestLayer
<rick_h_> that's what I've done in the past to run just JS tests.
<gary_poster> rick_h_, no, that's not good enough I'm afraid.  That's not an lxc thing, it is a server with no X running thing
<rick_h_> ah ok
<gary_poster> so, if you look in the tree...
<gary_poster> well, all you *have* to use you can find in the current .testr.conf: xvfb-run ./bin/test [other options]
<gary_poster> there's a more "careful" spelling we use for the tests
<gary_poster> xvfb-run --error-file=/var/tmp/xvfb-errors.log --server-args='-screen 0 1024x768x24'  -a bin/test blah blah blah
<gary_poster> the --error-file is the least important
<rick_h_> ok cool, yea I see that in the test_merge file
<rick_h_> test_on_merge that is
<gary_poster> rick_h_, that's server specific.  there is one lxc bug unfortunately
<gary_poster> that affects this
<gary_poster> lemme get that detail
<lifeless> StevenK: I asked jelmer to make an informed decision ;)
<rick_h_> ok thanks. I assumed it was a lxc issue
<lifeless> StevenK: so I will cheerfully a) disclaim all responsibility and b) buy you a beer anyhow.
<gary_poster> (we have a script that sets up all of this for you, but it is not ready for devs yet)
<rick_h_> understood, I really just want to make sure I didn't break the JS test runner atm, so if only a single test can run I'll be happy
<gary_poster> rick_h_, so, here's the pertinent code.  http://pastebin.ubuntu.com/1059579/
<gary_poster> basically it is saying that in the fstab file for the lxc instance,
<gary_poster> you need to add what is in line 7
<gary_poster> for bug 974584
<_mup_> Bug #974584: Semaphores cannot be created in lxc container <paralleltest> <patch> <rls-q-incoming> <Launchpad itself:Invalid> <lxc (Ubuntu):Fix Released by serge-hallyn> <sysvinit (Ubuntu):Triaged> <lxc (Ubuntu Precise):Fix Released> <sysvinit (Ubuntu Precise):Triaged> <lxc (Ubuntu Quantal):Fix Released by serge-hallyn> <sysvinit (Ubuntu Quantal):Triaged> <sysvinit (Debian):New> < https://launchpad.net/bugs/974584 >
<gary_poster> rick_h_, restart the lxc after you make that change
<rick_h_> orly, ok thanks
<gary_poster> If it doesn't work with xvfb-run and that workaround, I dunno, but you can feel free to ask me again :-)
<rick_h_> I think it's working, at least I've got my first test failures so I think just hte xvfb-run got me started
<rick_h_> ty much
<gary_poster> welcome
<lifeless> cjwatson: whats your lp username?
<czajkowski> sinzui: you're on a role!
<sinzui> czajkowski, no, my script is. I need to clean up bug data to tell stakeholders why a squad has work on disclosure for a year and still not delivered sharing or private projects
<czajkowski> sinzui: but you are close and have done a good job it's not an easy task a  lot of areas to make happy
<sinzui> czajkowski, I am not close.
<sinzui> There is a caustic streak of optimism at Canonical that misleads everyone into thinking we are always a few months away.
<sinzui> I currently believe we are 5 months away from delivering disclosure, which is 3 months longer than stakeholders have been told, and more than a year longer than they hoped we would take after a year of saying we would do it
<czajkowski> sinzui: ok but 5 is better than 8 and no point in saying 3 as if you say that and it's not tested or still has issues then people tend to get more upset, this way if you deliver before the 5 then people are happy, as i said it's a large project with many areas to keep happy
<lifeless> heh
<sinzui> czajkowski, I cannot prove what I say. When someone asks me what percentage is done. I cannot answer, I cannot even guess
<sinzui> I cannot even summarise all the phases of work someone can see why there is an order and doggedly slow pace to delivering private projects
<lifeless> sinzui: do you have a gut feel - underestimated? lots of waste? lots of friction? poor cycle times?
<czajkowski> sinzui: frustrating I know really, but I do feel you're doing a great job, and you blog posts are very good and they explain the things that are changing and what is coming next
<sinzui> under estimating is the primary cause. Edge cases, secondary use cases, hardening, removal of obsoleted code was often forgotten
<lifeless> would it have helped to resource and implement much smaller slices
<lifeless> ?
<sinzui> ^ lifeless. churn is less of a communication problem.
<sinzui> lifeless, yes, and doing them in the proper order or in parallel
<sinzui> bug linking should have started after picker 10 months ago
<sinzui> lifeless, Even cases like discovering we had to hard objects and reconsider social private teams were always something that could have been done by other teams.
<sinzui> Well I guess those cases were not discovery. We did know about them, we just did not know scope so we ignored them until it hurt
<cjwatson> lifeless: cjwatson
<lifeless> cjwatson: with great power comes great...
<lifeless> well you know
<cjwatson> cheese?
<lifeless> entertainment?
<cjwatson> voltage
<cjwatson> thanks anyway :)
<lifeless> rock n roll
<lifeless> (ac/dc)
 * cjwatson attempts to land copy-custom-uploads for about the sixth time today; this is cursed
<czajkowski> cjwatson: you do work long days
<lifeless> czajkowski: welcome to Canonical
<lifeless> czajkowski: we're a fun company of individuals.
<cjwatson> don't confuse elapsed time with actual time spent working though
<czajkowski> lifeless: ah ye are really, why else do you think I stuck around for so long to find a way in ;)
<czajkowski> cjwatson: true
<cjwatson> czajkowski: Also I find that LP's long EC2/buildbot cycles plus deployment schedules plus QA means that if I want to get work landed at the optimal speed then the best way to do it is to work skimming-stones-on-a-lake kind of hours
<czajkowski> cjwatson: ah so you do have a method t the madness :)
<lifeless> cjwatson: yes, OTOH that is getting fixed. Its tough shit though :)
<cjwatson> suits the new-parent lifestyle well enough though
<lifeless> indeedy
<cjwatson> (well, new-baby not new-parent, but it comes out the same way)
<lifeless> man, cynthia is teething right now
<cjwatson> yeah, that's no fun
<lifeless> 4 hours sleep last night for me :<
<lifeless> </whinge>
<cjwatson> andre is vaguely considering the notion of solid food
<czajkowski> :/
<lifeless> cjwatson: in a positive or not fashion?
<cjwatson> it's all right, bit earlier than we expected him to have a go but not a problem.  he's not really got into it properly, just vaguely flailing at stuff and occasionally a bit reaches his mouth
<cjwatson> but clearly on the road to figuring it out
<lifeless> excellent
<lifeless> thats better than nononononono I don'twanna. (mimed, of course)
<cjwatson> yeah, we do baby-led weaning so not much of that with food.  although plenty with other stuff ;-)
<lifeless> us too, but we've heard horror stories ;)
<cjwatson> judith nearly got killed on the road today though, kirsten came back shaking :-(
<cjwatson> got away from her and started running
<cjwatson> she reached her one step before the road
<czajkowski> :o
<czajkowski> cjwatson: scarey I'm sure
<lifeless> cjwatson: yeepers
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
<cjwatson> 2012-06-25 21:21:22 DEBUG   process-upload ran in 5.889692s (excl. load & lock)
<cjwatson> every time I read that kind of output I want it to say "lock & load"
<rick_h_> wallyworld_: looks like the non-min js landed a min ago. Give it a shot if you get a sec please.
<wallyworld_> rick_h_: awesome, thanks!
<rick_h_> wallyworld_: also note the second item here: https://docs.google.com/a/canonical.com/document/d/1jvZw_F8P0wXIxeNSpcEOpZbWqO0MSRZpn5Q3N2Mdcb8/edit
<wgrant> sinzui, wallyworld_: https://github.com/mattdiamond/fuckitjs/
<rick_h_> wgrant: :)
<rick_h_> wallyworld_: it's in a lot of the picker stuff in case you're still tinkering in there
<wallyworld_> rick_h_: yeah, i saw the naughty way of doing it a lot in the code and probably cargo culted it believing it was ok
<rick_h_> wallyworld_: yea, all good. Easy fix, but want to get the word out.
<wallyworld_> thanks, appreciate the heads up
<wallyworld_> rick_h_: works great \o/
<wallyworld_> rick_h_: you should send a post to the dev list so there can be much rejoicing
#launchpad-dev 2012-06-26
<cjwatson> https://dogfood.launchpad.net/ubuntu/precise/+source/cjwatson-pcj-testing/1.1 - qualified exclamation of glorious victory
<cjwatson> The Builds link goes to the private archive - do we think that's a problem?
<cjwatson> I think it's unrestricted the actual .debs though
<cjwatson> Let's see what publishing that does
<wgrant> cjwatson: That's normal.
<wgrant> Indeed, it looks good.
<cjwatson> Ah, that BPB is in fact public because BPB permissions are odd.
<wgrant> Right, BPB.private === all(spph.archive.private for spph in bpb.spr.spphs)
<cjwatson> So that's all good.
<StevenK> wallyworld, wgrant: https://code.launchpad.net/~stevenk/launchpad/destroy-old-privacy-ui/+merge/112002
<wallyworld> lots of LOC credit there
<StevenK> It only hits -488
<StevenK> Bug{Visibility,Security}Change are probably only worth another -20
<StevenK> Er, -200
<wallyworld> StevenK: field_names on BugSecrecyEditView doesn't need to be a property anymore
<wallyworld> same with schema i think
<StevenK> schema has to remain a property
<StevenK> I think you're right, field_names = ['information_type'] should be fine
<wallyworld> why does scheme need to be a property? lots of other places don't do it that way
<wallyworld> it was only a property because of the logic involved
<wallyworld> with the feature flag afaiui
<StevenK> wallyworld: It needs to protect the class defined in it to not get eval'd too early.
<StevenK> wallyworld: Keep in mind InformationTypeVocabulary() still has two feature flags that get read on initialize.
<wallyworld> ah, ok. thanks
<wallyworld> StevenK: these lines:
<wallyworld> 159	- if not self.request.is_ajax:
<wallyworld> 160	- return canonical_url(self.context)
<wallyworld> 161	- return None
<wallyworld> 162	+ return canonical_url(self.context)
<wallyworld> i don't think they need to (or can be) changed
<StevenK> We no longer do a AJAX call
<wallyworld> really? ok
<wallyworld> i guess we do a patch then?
<wallyworld> it must be an xhr call of some sort though
<StevenK> wallyworld: Sure, which is done in the JS
<StevenK> Doesn't involve the form or its next_url in any way
<wallyworld> you sure? many other places in the js do a post and reuse the same form at the backend
<wallyworld> so the ajax check is required
<StevenK> wallyworld: Yes. The JS uses ChoiceSource
<wallyworld> and this was originally written to work that way
<wallyworld> ok, so the choicesource widget must call lp_client.patch then i assume
<wallyworld> which does a named_post
<StevenK> Yes, which uses the *API*, not the form machinery.
<wallyworld> yes
<wallyworld> StevenK: so if the api is used, transitionToInformationType() returns True. the old ajax call returned the direct subscribers so the subscription portlet could be updated since changing info type also can mess with subscribers. how do we handle that now?
<StevenK> wallyworld: We don't, currently. But I'm not sure we care since changing the information_type isn't going to mess subscribers soon.
<wallyworld> StevenK: you sure? there's a card jc is looking at which will
<wallyworld> bug 1014922
<StevenK> wallyworld: The reason it updates subscibers is due to visibility, and the rule that subscription confers visibility is going away, so ...
<wallyworld> StevenK: but you are not allowed to be subscribed to bugs you can't see
<wallyworld> so if a bug is made private, some people don't be able to see the bug anymore, hence will be unsubscribed
<wallyworld> plus making a bug private may add subscribers in defined roles
<wgrant> No
<wgrant> Well, only the person who is performing the action
<wgrant> It won't add others.
<wallyworld> defined roles definition may change sure
<wallyworld> atm it adds bug supervisor or pillar owner etc
<wallyworld> but the point is, the subscribers list changes and we seem to have a regression
<wallyworld> in not updating the ui after a info type change
<wallyworld> since before the ui change, it all worked as expected
<wallyworld> StevenK: i think we can delete this entire block of code?
<wallyworld> 371	- if (not bool(getFeatureFlag(
<wallyworld> 372	- 'disclosure.show_information_type_in_ui.enabled')) and
<wallyworld> 373	- extra_data.private):
<wallyworld> 374	- if params.information_type == InformationType.PUBLIC:
<wallyworld> 375	+ if extra_data.private:
<wallyworld> 376	+ if params.information_type in PUBLIC_INFORMATION_TYPES:
<StevenK> Nope, it's used, I think.
<StevenK> There were one or two tests that depended on the behaviour at least.
<wallyworld> StevenK: but won' the initial if condition in the original code always be false with the ff turned on? so the code will never be evaluated
<StevenK> wallyworld: I can't recall the reason I added the guard there.
<StevenK> It clearly shouldn't be.
<StevenK> I think I had the guard when that function was expecting private/security_related to flow through the whole thing
<wallyworld> perhaps with ff turned on extra_data.private is replaced by extra_data.information_type
<StevenK> wallyworld: Sadly not
<StevenK> extra_data.information_type should probably be added.
<wallyworld> yes
<wallyworld> StevenK: what text is printed out in place of this?
<wallyworld> 843	- >>> print admin_browser.contents
<wallyworld> 844	- <!DOCTYPE...
<wallyworld> 845	- ...This is a private bug...Foo Bar...won&rsquo;t be notified...
<wallyworld> do we need to fix the test so it checks?
<StevenK> That is looking for a notification that is no longer printed.
<wallyworld> ok
<StevenK> wallyworld: That test sucked to debug. The notification isn't in the contents so the doctest prints out the 400 lines of contents
<wallyworld> yeah, i hate that too
<wallyworld> StevenK: in this case though, can't we check for the new info type text?
<wallyworld> 985	- ... "This bug report should be private").selected = False
<StevenK> wallyworld: Ah, we're setting it back so we don't fuck up the next test in the doctest, no longer needed.
<wallyworld> of course, silly me
<wallyworld> StevenK: r=me with a comment to raise a bug for the regression
<StevenK> wallyworld: Right, okay. But that bug might end up being marked as Invalid :-)
<wallyworld> what am i missing?
<wallyworld> the subscribers list can change, no?
<wallyworld> at the very least, the person marking the bug private will be subscribed
<wallyworld> so the subscribers list will need updating
<StevenK> wallyworld: Oh, your second comment is related to extra_data.private -- there's a test around for that -- I'll ignore that bit for now, and mention to Curtis if it is actually used at all
<wallyworld> ok
<wallyworld> the second one was more of a request just to see what might be needed
<StevenK> wallyworld: https://bugs.launchpad.net/launchpad/+bug/1017818
<_mup_> Bug #1017818: subscriber list does not update after changing information type <disclosure> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1017818 >
<StevenK> Oh sure, *now* _mup_ answers.
<wallyworld> StevenK: thanks. i hope i'm not missing something stupid asking for that bug to be raised
<StevenK> wallyworld: Well, as you say, the person doing the action may end up subscribed
<wallyworld> and *possibly* others removed
<wallyworld> as per jc's card
<StevenK> Right
<StevenK> Stuff might happen, as it were.
<wallyworld> StevenK: so we may need to undelete the code that constructs the json data struct :-)
<StevenK> Ew
<StevenK> It will probably have to be in JS, rather than in the form
<wallyworld> well, ideally we wouldn't make a 2nd xhr call
<wallyworld> so the call to update the info type would return the data needed to update the ui
<StevenK> Maybe.
<wallyworld> or at least update the request cache
<StevenK> That sounds okay
<wallyworld> which would also eliminate a 2nd call
<wallyworld> whatever works :-)
<StevenK> wallyworld: This branch has been through ec2 twice, but since it's so large I'm going to toss it for a third run.
<wallyworld> good idea :-)
<jelmer> 'morning
<jelmer> jam!
<jam> jelmer: !
<jam> cjwatson: you're on the blame list for buildbot failing. http://lpbuildbot.canonical.com/builders/lucid_lp/builds/2182
<jam> It looks unrelated (celery failure,  I believe)
<cjwatson> Yeah, no way that's related to my change.  -ops suggested it's already been forced
<jelmer> gmb: https://code.launchpad.net/~jelmer/launchpad/no-revhistory-1/+merge/112018
<adeuring> good morning
<StevenK> cjwatson: Yes, twice, in fact.
* gmb changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: gmb (sprinting, so YMMV) | Firefighting: - | Critical bugs: 3.47*10^2
<jelmer> gmb: http://paste.ubuntu.com/1060456/
<gmb> jelmer, https://lists.launchpad.net/yellow/msg00938.html
<jelmer> gmb: gracias
<wgrant> adeuring: That patch should land on devel if it's been applied live.
<wgrant> adeuring: Not db-devel
<wgrant> adeuring: That is a live patch, so it's probably been applied live already or should be soon.
<adeuring> wgrant: no, flacoste asked me to back it out.
<adeuring> wgrant: and since the bug is really old, it does not matter if the brach is merged a few days sooner or later ;)
<wgrant> adeuring: I saw, but I think he misunderstood.
<wgrant> https://dev.launchpad.net/PolicyAndProcess/DatabaseSchemaChangesProcess
<wgrant> "For hot patches the target is lp:launchpad, but for cold patches it should be lp:launchpad/db-devel. "
<wgrant> And that looks like a hot patch
<wgrant> stub: Has the ftq patch been applied anywhere yet?
<stub> wgrant: no, not that I'm aware of
<stub> IIRC it is a hot patch, no need for downtime
<adeuring> wgrant: yes, you're right -- but I'll simply merge the branch now into db-devel ;)
<wgrant> Right, => devel
<wgrant> adeuring: No!
<wgrant> db-devel is for cold patches
<adeuring> why?
<stub> I don't care where it lands but I'm not the shephard
<wgrant> If you merge it into db-devel, all it means is I have to merge db-stable into devel an extra time tomorrow
<wgrant> It achieves nothing other than delays :)
<wgrant> db-devel is the downtime queue
<wgrant> This doesn't need downtime, so it does not belong on db-devel.
<adeuring> wgrant: so just wait until a merge is necessary anyway ;)
<wgrant> Simplest solution is to ask stub to apply it now and then land to devel in 5 minutes when it's applied everywhere :)
<stub> point me at the patch/mp
<adeuring> stub: https://code.launchpad.net/~adeuring/launchpad/bug-29713/+merge/111212
<adeuring> tjhough that one is meanwhile reverted
<adeuring> stub: https://code.launchpad.net/~adeuring/launchpad/bug-29713-db-dev/+merge/112043
<stub> adeuring:  I'll apply the db patch in https://code.launchpad.net/~adeuring/launchpad/bug-29713-db-dev/+merge/112043 now.
<adeuring> stub: great, thanks!
<stub> Gah. Whitespace munging, cut & paste, urgh
<stub> adeuring: done
<adeuring> stub: thanks
<adeuring> stub: you ran the patch on qastaging, right?
<stub> adeuring: No, just production
<stub> I'll do qastaging now
<adeuring> stub: ok
<stub> Done on qastaging
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: gmb (sprinting, so YMMV), rick_h | Firefighting: - | Critical bugs: 3.47*10^2
<cjwatson> wgrant: https://code.launchpad.net/~cjwatson/launchpad/remove-single-custom-upload-exception/+merge/112093
<wgrant> cjwatson: Thanks
<ivory> rick_h_: abel said you could maybe help me with a problem in html (if you got time for it)?
<rick_h_> ivory: sure thing
<ivory> rick_h_: http://bazaar.launchpad.net/~ivo-kracht/launchpad/bug-806660/revision/15436
<ivory> rick_h_: i created a seperate form but the "Add" button is offset and i dont know why
<ivory> rick_h_: oh this is the bug i amworking on https://bugs.launchpad.net/launchpad/+bug/806660
<_mup_> Bug #806660: "Add a new address" in e-mail settings does the wrong thing when pressing Enter <easy> <ui> <Launchpad itself:In Progress by ivo-kracht> < https://launchpad.net/bugs/806660 >
<rick_h_> loading
<rick_h_> ivory: so the add button is offset in the initial html? You're wanting to get rid of that offset?
<ivory> rick_h_: no the new one is offset to the right and i want it to be like theold one was
<rick_h_> ivory: so what I would do is colspace the <td> and then using padding-left to get it where you want it.
<rick_h_> ivory: now that it's not connected to the rules of the other form, it won't really line up automatically and it's splitting the width 50/50 usually by default
<rick_h_> the wide content of the second <td> in the upper form was pushing the button over for you
<rick_h_> honestly, they should probably be defined in some way so that it's consistant. So with a common width set for that first <td> on both forms
<ivory> ah ok i just wondered why it did that, thank you for helping
<rick_h_> np
<sinzui> rick_h_, Do you agree with my assessment...should this bug be marked wontfix? https://bugs.launchpad.net/launchpad/+bug/383943
<rick_h_> loading
<rick_h_> sinzui: right, it's the API
<rick_h_> sinzui: I mean we could, via getter/setter ATTRS try to mirror things for easier API usage
<rick_h_> but seems like more work than it's worth
<Laney> can I haz database patch number for bug #1016776 please?
<_mup_> Bug #1016776: Users are offered updates to packages in the -proposed staging area pre-release <feature> <launchpad> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1016776 >
<cjwatson> Argh, I hate DB security
<rick_h_> cjwatson: so ummm, I assume this branch is part of stuff you've been working with the ppa pros on?
<rick_h_> cjwatson: so you need this to land by wed in production? or the prev code is scheduled to be tested there then?
<cjwatson> rick_h_: Which one, sorry?
<rick_h_> cjwatson: sorry, the one in review with your deletions for the unembargo code
<rick_h_> https://code.launchpad.net/~cjwatson/launchpad/remove-unembargo-package/+merge/112044
<cjwatson> rick_h_: No, I was noting an intent not to land it until after I've made sure that the ubuntu-security team can use the new code that should hopefully be deployed tomorrow
<cjwatson> There's no rush on remove-unembargo-package; was just getting it in for review early, that's all
<rick_h_> ok, so I'll make a comment that review will pend the test
<cjwatson> Right, that's what I thought I'd written in my cover letter :-)
<cjwatson> Well, there's no harm reviewing it earlier
<rick_h_> right, well you said you won't land it, but as the reviewer I'll punt it until after entirely if that doesn't block you at all
<cjwatson> But sure
<rick_h_> it's not exactly my wheel house, but I don't want to hold you up if you needed this
<cjwatson> Not at all, was just getting stuff off my disk
<rick_h_> ok, cool
<rick_h_> Laney: let me pull that down and get you an id
<cjwatson> What is a bit more urgent is my r15491, which I've just marked qa-bad; should I revert that?  I think it's just a security.cfg patch to fix it, but I do need to figure out a test
<rick_h_> cjwatson: hmm, so I don't think there will be a deploy by us today. So if you can get the fix in I'd go for the fix
<rick_h_> if it'll be tomorrow or tonight (when the aussies get going) I'd revert it out
<cjwatson> Should be soon - couple of phone calls coming up but let's see what I can do
<rick_h_> cjwatson: ok, sounds like a plan
<rick_h_> sinzui: with that nth child, it's just grabbing the submit button right? I wonder if it could just be changed to grab based on name/etc? Or am I misreading that
<sinzui> rick_h_, yes, name would also work in this case.
<rick_h_> ah, they don't have names so that they're not submitted I'd imagine
<sinzui> oh, right <button>
<rick_h_> but they have classes I think he could use, so commenting.
<rick_h_> thanks for bringing that up
<sinzui> I really need to fix the button css so that it looks more like a platform button
<Laney> rick_h_: thanks
<jelmer> gmb: https://code.launchpad.net/~jelmer/launchpad/no-revhistory-2/+merge/112119
<jcsackett> sinzui: free to chat a bit?
<sinzui> yes
<jcsackett> cool.
<jcsackett> sinzui: i've started a hangout.
<rick_h_> Laney: ok, claimed 2209-25-1 for you
<Laney> excellent, ty
<jelmer> gmb: https://code.launchpad.net/~jelmer/launchpad/no-revhistory-3/+merge/112123
<cjwatson> rick_h_: https://code.launchpad.net/~cjwatson/launchpad/copy-custom-uploads-fix/+merge/112122 fixes that regression
<rick_h_> cjwatson: looking now
<rick_h_> cjwatson: ok, r=me
<cjwatson> Thanks
<cjwatson> I'll get that landed then
<cjwatson> And hopefully re-QA that tonight
<ivory> rick_h_: could you review this? https://code.launchpad.net/~ivo-kracht/launchpad/bug-806660/+merge/112125
<rick_h_> sure thing ivory
<ivory> rick_h_: thank you
<sinzui> jcsackett, http://pastebin.ubuntu.com/1060851/
<rick_h_> ivory: r=me
<rick_h_> flacoste: so part of this yui gallery bit is signing their CLA. Should I just sign as an individual, or can I sign this as "Canonical" the organization?
<flacoste> do they have a formal process for organization?
<flacoste> i don't think it's possible
<rick_h_> it's the same form, just with the org filled in and my title added
<flacoste> for example, for the Canonical CLA, only individuals can sign them
<rick_h_> but there is a checkbox to choose so I wanted to make sure I was going this the right way
<rick_h_> https://secure.echosign.com/public/hostedForm?formid=A9PFU5T58653A
<rick_h_> ok, I can do individual without worry if that's cool and easier to do
<flacoste> yep
<flacoste> i think that's the best move forward
<rick_h_> ok, thanks
<Laney> Just noticed checkmarkable on the dev wiki. I can't use it (forbidden). Should I be able to? :-)
<cody-somerville> Why is it taking so long for PPAs to publish built binaries these days?
<czajkowski> sinzui: what is the difference of the tag hardening and say privacy ?
<sinzui> hardening was a phase of work where we wanted to control *who* has permission. privacy is largely about *what* is and is shown as private
<sinzui> hardening is a feature tag.
<czajkowski> sinzui: ah ok, thank you
<mgz> gmb: https://code.launchpad.net/~gz/launchpad/py27_test_scriptmonitor_logging_1017981/+merge/112160
<gmb> mgz: https://dev.launchpad.net/EC2Test
 * czajkowski waves at gmb 
<czajkowski> gmb: working without you is rather quiet
<czajkowski> :s
 * gmb waves at czajkowski 
<gmb> Haha.
<gmb> Eh, well, I'll be in the office for an afternoon Thursday fortnight, if that's any help.
<czajkowski> yarp can be indeed
<czajkowski> going in Friday this week as there is after work drinks
<czajkowski> maxb: cjwatson may know
<rick_h_> gmb: can you review this please? https://code.launchpad.net/~rharding/launchpad/yui35_test/+merge/111911
<rick_h_> gmb: if the sprint has you busy I'll bug deryck next :)
<czajkowski> rick_h_: I think he's EOD he's sprinting
<rick_h_> czajkowski: yea, why I ping'd. Didn't update topic/is still in irc, but expecting to bug deryck in a sec
<deryck> rick_h_, I can do a review here shortly.  About to do call with abentley
<rick_h_> deryck: rgr, no rush. Thank you
* czajkowski changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: rick_h | Firefighting: - | Critical bugs: 3.47*10^2
<cjwatson> czajkowski: hmm?
<lifeless> sinzui: have we had any more user confusion about 'contact this user' since the last round of tweaks ?
<sinzui> lifeless, The only remark on irc and bugs that I have seen was from our own abentley regarding mass team admin emails.
<sinzui> lifeless, I think this is a team hierarchy issue.
<sinzui> maas team, not mass team
<lifeless> cool
<lifeless> sounds like we might have put the monster to rest
<maxb> cjwatson: I was wondering on #launchpad if there was a way to ask LP for which Ubuntu series it currently accepted PPA uploads
<maxb> czajkowski moved the conversation here thinking you more likely to be around here :-)
<abentley> lifeless: Are you aware of any issues with codehosting that prevent opening more than 4 branches in rapid succession?  I see it on qastaging and launchpad.dev, but not production.
<maxb> Although.... something seems to have changed in that department, since LP just accepted an upload for edgy (and then failed to build it, of course)
<cjwatson> I believe (from the code) that PPAs allow you to upload to any suite that exists, probably because of PES.
<cjwatson> edgy is a bit weird; edgy-* exist on ftpmaster.internal but not edgy itself
<cjwatson> You can certainly upload to anything currently supported or in-development by Ubuntu.  Anything else is probably best regarded as at-risk.
<cjwatson> (But I'm not really a PPA guru, this is just my impression)
<cjwatson> As it happens at the moment I rather suspect dapper + feisty-quantal should work provided that the distroarchseries has a working chroot_url; but we've been talking about clearing some of those dists subdirectories off ftpmaster at some point, which will break that.
<lifeless> abentley: bzr+ssh ?
<lifeless> abentley: or sftp, or $unknown ?
<lifeless> abentley: I'm not aware of any intrinsic limits, no. are the opens serialised, or do you have 4 clients concurrently hitting it ?
<abentley> lifeless: bzr+ssh and probably sftp
<abentley> lifeless: Serialized.
<abentley> lifeless: https://pastebin.canonical.com/68914/
<lifeless> abentley: whats the failure you get ?
<abentley> lifeless: No explicit failure.  It hangs after "bzr+ssh://bazaar.qastaging.launchpad.net/+branch-id/8633".
<abentley> lifeless: When I run it locally, it works.  But on devpad, it fails.
<abentley> i.e. hangs.
<lifeless> do you have ssh master connection use on ?
<lifeless> lsof / netstat / strace may help you determine what it is hanging on
<lifeless> abentley: if you drop into pdb (ctrl-\ IIRC), what is bzr trying to do ?
<abentley> lifeless: No, I don't have ssh master connection use on.
<abentley> lifeless: This is a script, so ctrl-\ doesn't drop to pdb.
<lifeless> abentley: import bzrlib.breakin; bzrlib.breakin.hook_debugger_to_signal()
<lifeless> abentley: should enable that
<abentley> lifeless: Thanks.  Backtrace: http://pastebin.ubuntu.com/1061439/
<lifeless> its waiting for ssh to exit
<lifeless> hmm
<cjwatson> wgrant,StevenK: Dear NDT cabal: I've QAed my follow-up fix to bug 231371 (rick_h advised that I could land that rather than reverting as long as I was quick enough that it'd be in devel by the time you folks woke up).  QA notes in the bug.  I haven't marked it qa-ok yet because (a) it's not on qastaging and (b) there are several revisions between the bad one and the fix, so I guess you need to be aware not to deploy ...
<_mup_> Bug #231371: support dist-upgrader-all pocket copy <feature> <lp-soyuz> <qa-bad> <soyuz-publish> <Launchpad itself:Fix Committed by cjwatson> < https://launchpad.net/bugs/231371 >
<cjwatson> ... r15491 without r15499.
<cjwatson> It'd be nice if at least up to r15489 could be deployed tomorrow, since I have a window with the security team tomorrow to production-test r15486.
<lifeless> abentley: sorry am OTP will ping you soon
<lifeless> abentley: so, ssh isn't exiting
<lifeless> abentley: we've seen this before I think; one common case is master connections, but also..
<abentley> lifeless: The function assumes that closing stdin/stdout will terminate ssh.
<abentley> lifeless: in recent versions of bzr, a socket is used.
<lifeless> abentley: I'd poke around with lsof on the ssh process its waiting for
<lifeless> abentley: see what it thinks is up
<abentley> lifeless: It still has a tcp connection: "TCP localhost.localdomain:44533->launchpad.dev:5022 (ESTABLISHED)", and two sockets.
<abentley> lifeless: http://pastebin.ubuntu.com/1061512/
<bac> hi james_w, did you get my email request regarding your tarmac scripts?  if not, i'd really like to have a look if you have time to sanitize and send them.
<james_w> bac, I did
<james_w> bac, I
<james_w> 'll sanitise now
<bac> james_w: thanks a lot!
<timrc> Any idea how I get a list of persons subscribed to a private PPA via the api (akin to +subscriptions in the web ui)?
<timrc> I'm seeing https://launchpad.net/+apidoc/devel.html#archive_subscriber, but not sure how to access it
<james_w> bac, lp:~james-w/+junk/tarmac-puppet
<bac> got it james_w, thanks
<wgrant> jelmer: Have you tested that this Branch.revision_history excision attempt will be less catastrophic than the previous one?
<wgrant> cjwatson: Ergh, OK
<jelmer> wgrant: Yes - it's also less ambitious
<jelmer> wgrant: this one doesn't try to improve performance in any way, it just copes with the fact that bzr has deprecated these two functions but keeps O() the same
<jelmer> wgrant: in other words, it won't help with bug 808930
<_mup_> Bug #808930: Timeout running branch scanner job <escalated> <linaro> <oops> <Launchpad itself:In Progress by abentley> < https://launchpad.net/bugs/808930 >
<wgrant> jelmer: Right, sounds good. Thanks.
<jelmer> I do want to improve that bit some time, but not now :)
<sinzui> wgrant, https://dev.launchpad.net/Projects/Disclosure
<sinzui> StevenK,  https://dev.launchpad.net/Projects/Disclosure
<cjwatson> timrc: Right now, I don't think you can.  ArchiveSubscriber only seems to be exported as the return type of Archive.newSubscriber.
<cjwatson> *newSubscription
<timrc> cjwatson, :/
<rick_h_> huwshimi: heads up, got permission to submit the morph module to the YUI Gallery so submitted that today.
<huwshimi> rick_h_: Awesome! Nice work :)
<rick_h_> huwshimi: I hear you wrote most of it, I've got a branch of it here: https://github.com/mitechie/yui3-gallery/tree/add_morph/src/gallery-anim-morph
<rick_h_> updated to all YUI specs and all that jazz. So if we go to reuse it in the future (once this gets approved/merged) it'll be considered the upstream to sync up with.
<huwshimi> rick_h_: Nice one :)
<huwshimi> rick_h_: I'll have to make sure we replace it in MAAS too.
<rick_h_> yea, hopefully going to work on getting more generic stuff up into the gallery
<rick_h_> and try to keep apps sync'd using hte gallery versions of things
<rick_h_> huwshimi: yea, at some point we can look at that. This adds an API chance we needed for the other project and some more tests, docs, etc.
#launchpad-dev 2012-06-27
<wgrant> danilos: Hi, what's happening with https://code.launchpad.net/~danilo/launchpad/bug-1000148/+merge/105962?
<StevenK> wgrant: http://pastebin.ubuntu.com/1061930/
<wgrant> StevenK: :(
<wgrant> StevenK: the methods take iterables for a reason :)
<wgrant> Precisely to avoid loops like that.
<StevenK> wgrant: I need the artifact anyway, so .ensure() doesn't get out of it
<wgrant> StevenK: Hm?
<wgrant> ensure() returns all the AAs.
<wgrant> So one call can get all the AAs for the batch
<StevenK> wgrant: Sure, but then I need to match up the AA to the branch
<wgrant> StevenK: that's roughly one generator expression away
 * wgrant disables test_find_missing_ready
<StevenK> Oh, is that the one that keeps failing horribly on buildbot, but passing on ec2?
<wgrant> Yes.
<wgrant> Not reproducible locally.
<wgrant> But happens fairly often on buildbot.
<StevenK> wgrant: Does http://pastebin.ubuntu.com/1061964/ make you happier?
<wgrant> StevenK: Still O(chunksize) queries, but only for branch.subscriptions for which there is no batch API, so I no longer hate you.
<StevenK> Yeah, and I'm not writing one just for a garbo job. :-P
<StevenK> Laney: Please set a description for your MP
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/populate-branch-aag/+merge/112264
<wgrant> StevenK: Looks mostly OK, but we still don't know what we want to do about owners
<StevenK> wgrant: What else did I have to sort out?
<wgrant> StevenK: Well, we need to think about owners.
<wgrant> Because they should have access except when they should not.
<StevenK> Oh, I need to fix the job stuff to deal with branches too?
<wgrant> Right, that's something that only touches on ownership, so it's probably sensible to work on now.
<StevenK> Oh good, it's SharingJob, so I don't have to kill wallyworld.
<StevenK> """Job classes related to the sharing feature are in here."""
<StevenK> On second thoughts ...
<StevenK> wgrant: I can recall their being two, is RemoveBugSubscriptionsJob the winner?
 * StevenK stabs buildbot more
<wgrant> StevenK: There's now only one.
<wgrant> They were merged into RBSJ
<StevenK> So I should write a seperate RBranchSJ?
<wgrant> StevenK: Possibly. I don't know if it's better to duplicate or extend in this case.
<StevenK> wgrant: Extending involves a rename :-(
<wgrant> StevenK: Is that a problem?
<wgrant> I don't think the name is stored
<wgrant> Just an integer.
<StevenK> Right, not really
 * StevenK greps for 'are in here' and sighs
<StevenK> wgrant: RBSJ -> RemoveArtifactSubscriptionsJob ?
<wgrant> StevenK: Maybe
<StevenK> wgrant: You seem very unsure these past two days :-P
<wgrant> If I was sure of the solutions they'd be done already :)
<StevenK> wgrant: Oh, what's the plan to keep AAG up to date WRT branch after the garbo job?
<StevenK> The job will remove subscriptions, but what adds and removes AAGs?
<wgrant> StevenK: See Bug.subscribe and Bug.unsubscribe
<StevenK> Ah, _reconcileAccess
<StevenK> We need one for branch, no?
<wgrant> No
<wgrant> _reconcileAccess is separate
<wgrant> It maintains AA and APA
<wgrant> And is already in place for Branch.
<StevenK> Oh, service.ensureAccessGrants()
<StevenK> Both Bug and Branch make use of that
<wgrant> Hm
<wgrant> wallyworld must have already done that bit
<wgrant> Sneaky
<StevenK> I did the branch bit
<wgrant> It's not done in unsubscribe, though.
<StevenK> ... Details? :-)
<wgrant> StevenK: Bug.unsubscribe removes grants, Branch.unsubscribe does not
<StevenK> wgrant: Yes, I get that, I was making a joke.
<StevenK> wgrant: In either case: http://pastebin.ubuntu.com/1062089/
<wgrant> StevenK: Probably, yeah.
<StevenK> wgrant: Would you like to review https://code.launchpad.net/~stevenk/launchpad/branch-unsubscribe-revokes/+merge/112272 ? Happy enough to leave it for someone else to pick up.
<wgrant> StevenK: Juggling three other things right now, so I might not get to it today.
<StevenK> wgrant: Like I said, happy enough to leave it.
<jam> lifeless: do you have any time to chat about bug #879589 ?
<_mup_> Bug #879589: adding/removing user/team subscriptions to bug reports edits date_last_updated <bughistory> <escalated> <ubuntu-qa> <Launchpad itself:Triaged> < https://launchpad.net/bugs/879589 >
<lifeless> jam: not tonight really
<lifeless> jam: perhaps tomorrow?
<jam> lifeless: k, prob means we won't be working on it this week, then.
<jam> lifeless: If not today, then any time works
<jam> we'll just get to it on rotation.
<lifeless> If I have time later tonight I'll ping you
<mgz> so, my ec2 land from yesterday finished without sending me email
<mgz> jml: you had an issue like this at the start of the month, do you remember if you changed something to get it working?
<jam> lifeless: np
<mgz> okay, it seems that not having my password in bazaar.conf makes it break
<mgz> using the canonical smtp server (which has a global password) is what works of most people
<wgrant> mgz: Right, the ec2 instance isn't going to have a huge amount of luck authenticating without knowing the password.
<mgz> checking that before setting up might be... nice
<danilos> wgrant, on https://code.launchpad.net/~danilo/launchpad/bug-1000148/+merge/105962, I am waiting for skaet (Kate Stewart) from Ubuntu to give me a go-ahead to land it (as if they are having second thoughts for Ubuntu)
<wgrant> danilos: You've not heard anything?
<adeuring> good morning
<danilos> wgrant, I have a call on Friday to settle that once and for all
<danilos> wgrant, will likely land then
* frankban changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: rick_h, frankban* (rvba) | Firefighting: - | Critical bugs: 3.47*10^2
<wgrant> danilos: Great, thanks.
<Laney> StevenK: ack, done, thanks
<czajkowski> morning
<jam> stub: Is there a good way to stop a psql script based on a query?
<jam> I'm looking here: If your query requires result count verification, write that into your script.
<jam> here: https://wiki.canonical.com/InformationInfrastructure/OSA/RequestLogging/LP/SQL
<jam> and it says the above line.
<jam> In this particular case, there is a count which should be 0
<stub> Can anyone in soyuz land come up with a better name for https://code.launchpad.net/~laney/launchpad/db-proposed-not-automatic-pre-release/+merge/112134 ?
<stub> jam: If you start needing anything advanced, switch to a Python script.
<jam> stub: well, I want a while loop, and a stop-if-not-zero. It seems a bit much to go to a python script for that
<jam> but certainly I could.
<stub> jam: The facilities you are looking for are there in pl/pgsql but then you are running in a single transaction
<wgrant> stub: My suggestion would be to just make it proposed_not_automatic
<wgrant> stub: The pre-release bit can be handled by unsetting the flag on release :)
<Laney> There is a separate suggestion from mpt to have this behaviour post-release too. It's not clear if we want it, but if we do then the two wouldn't necessarily be conflated.
<stub> jam: It doesn't have to be a pretty python script. Import psycopg2, connect, cur.executes() & cur.commits()...
<stub> con.commit I mean
<jam> stub: sure
<jam> I think that is fair enough
<stub> jam: You can also cheat. Assuming your statement is a noop if you repeat it a few hundred times with no data to change, just cut and paste it a few hundred times... but there are no constructs apart from basic variable substitution in psql scripts - it is designed as an interactive interpreter.
<jam> stub: np, thanks
<jam> it just seems so close
<stub> yes. I think future versions of PG are looking at solving this by allowing stored procedures and anonymous code blocks access to the transaction machinery - at the moment, they are always invoked inside a transaction.
<stub> jam: cursor.rowcount lets you know the number of rows affected by the last execute btw. It is the most convenient way of terminating a loop like I suspect you are writing.
<jam> stub: thanks. The first thing is actually to check that there is nothing 'pending', but the second thing is to loop until empty
<mgz> is this normal looking for an ec2 land in progress?
<mgz> <http://ec2-23-20-98-100.compute-1.amazonaws.com/summary.log>
<cjwatson> mgz: Doesn't load for me; it might be firewalled.
<cjwatson> mgz: The summary.log for my run currently in progress looks like this: http://people.canonical.com/~cjwatson/tmp/summary.log
<cjwatson> Is there any plan for an NDT deployment today?
<mgz> cjwatson: ah, sorry, hadn't realised the secgroup was set to allow only from ip block
<mgz> thanks for the example to compare against
<wgrant> cjwatson: There were a few buildbot failures.
<wgrant> cjwatson: Hopefully tomorrow
<cjwatson> wgrant: Oh, blast.  I'll have to reschedule with security
<cjwatson> Ah well
<wgrant> cjwatson: Well, let's see
<wgrant> cjwatson: buildbot will be done in 10 minutes.
<wgrant> Let's see where QA is...
<cjwatson> Mm, there's a bunch of bzr stuff to QA
<wgrant> Ah yeah
<wgrant> And that's the stuff that's broken production once already
<cjwatson> You always want a completely clear report, rather than just a clear head?
<wgrant> cjwatson: You don't need the custom upload stuff?
<cjwatson> Not with any particular urgency
<wgrant> Then let's deploy now.
 * wgrant deploys now
<cjwatson> In fact if that lands tomorrow it can get the copy-only-copyable-custom-uploads branch as well so it will OOPS less (harmlessly, but still).
<wgrant> Yeah.
<wgrant> YAY
<wgrant> buildbot failed again
<cjwatson> FFS
<wgrant> For a different reason, though
<wgrant> Since I disabled the usual intermittent test
<cjwatson> That freaked me out for a moment since I have a branch sitting around locally (but not pushed anywhere) that goes near that test.
 * wgrant forces, will convince ops to pull it manually tomorrow.
<wgrant> Yeah, I was going to blame you but then realised all your recent changes there were well past buildbot.
<bac> hi diogo
<matsubara> bac, hi
* rick_h_ changed the topic of #launchpad-dev to:  http://dev.launchpad.net/ | On call reviewer: frankban* (rvba) | Firefighting: - | Critical bugs: 3.47*10^2
<rick_h_> frankban: rvba if you get a chance to peek at https://code.launchpad.net/~rharding/launchpad/listingnav_yui35/+merge/112334 please
<rick_h_> frankban: rvba sorry for the line count, but only the top of the files were changed, the main body is indenting/moving each test suite to tests.
<rick_h_> so it looks scarier than it is
<rick_h_> only actual tests change is the addition in the tearDown() noted in the MP comments
<frankban> rick_h_: on a call now, will look ASAP
<rick_h_> frankban: no rush, ty much
<Laney> hmm
<Laney> does launchpad-dev discard or silently moderate mail from non-subscribers?
<wgrant> Laney: Launchpad mailing lists in general silently drop emails from addresses that aren't registered in Launchpad
<wgrant> I don't see anything in the moderation queue
<Laney> wgrant: I sent it from laney@u.c.
<Laney> <20120627103638.GB1489@raleigh>
<wgrant> Laney: ./vette:Jun 27 10:36:22 2012 (14313) Discarding forged message-id: <20120627103638.GB1489@raleigh>
<wgrant> Odd
<wgrant> Laney: Huh
<wgrant> Laney: That seems to trigger only if the sender address is the list address.
<wgrant> https://lists.launchpad.net/launchpad-reviewers/msg01120.html has the relevant diff.
<wgrant> That's the first time that message has shown up in a long time.
<Laney> Huh, indeed.
<Laney> Date: Wed, 27 Jun 2012 11:36:38 +0100
<Laney> From: Iain Lane <laney@ubuntu.com>
<Laney> To: launchpad-dev@lists.launchpad.net
<wgrant> Laney: Can you try again, maybe Bccing me so I can see what's actually happening?
<Laney> Jun 27 11:36:10 cripps postfix/cleanup[2751]: 14CCB20424: message-id=<20120627103638.GB1489@raleigh>
<Laney> Jun 27 11:36:11 cripps postfix/qmgr[13900]: 14CCB20424: from=<laney@ubuntu.com>, size=4401, nrcpt=1 (queue active)
<Laney> wgrant: Sure, doing now.
<Laney> wgrant: Done. (Bcc: â¦@c.c)
<wgrant> Laney: Oh
<wgrant> Laney: launchpad-dev is in the Reply-To that you sent. That might be relevant
<Laney> could be
<Laney> It's fairly standard practice to do that though.
<wgrant> Evidently not LP mailing lists :/
<wgrant> Mailman/Defaults.py.in:SENDER_HEADERS = ('from', None, 'reply-to', 'sender')
<wgrant> That is a little odd.
<Laney> :/
<mgz> said test kipple: http://pastebin.ubuntu.com/1062517/
<Laney> wgrant: I'll just send it without reply-to.
<czajkowski> Laney: join the list, just modered your mail
<Laney> czajkowski: I'd rather not get any more email :P
<Laney> You could whitelist me if you like
<mgz> gmb: https://code.launchpad.net/~gz/launchpad/py27_test_scriptmonitor_logging_1017981/+merge/112160
<sinzui> czajkowski, Laney, you are white listed a day after your message is approved from the moderation queue
<Laney> ack, cheers
<ivory> frankban: could you take a look at this? https://code.launchpad.net/~ivo-kracht/launchpad/bug-824292/+merge/112352
<frankban> ivory: I am doing another review, I will be happy to review your branch if you can wait a couple of minutes
<sinzui> czajkowski, look at this page: https://launchpad.net/~laney/+review
<ivory> frankban: no problem,take your time
 * Laney wonders what's on that page â¦
<czajkowski> sinzui: oh yes
<czajkowski> sinzui: not seen that been used before good to know
<czajkowski> cheers
<sinzui> czajkowski, standing is an incomplete feature. it was to permit non-subscribers to lurk and posts to lists.
<czajkowski> sinzui: ah interesting
<sinzui> czajkowski, Lp will notice laney's approved moderation later today and change something so that laney can always post to our list...
<czajkowski> ph I just changed it manually :/
<sinzui> ...but changing his standing to good will permit him to post to any Lp list I believe
<sinzui> standing is probably a candidate to remove since no one has commitment to finish it
<czajkowski> grand changed it to that
<gmb> jam: lp:~yellow/charms/oneiric/buildbot-master/trunk and lp:~yellow/charms/oneiric/buildbot-slave/trunk
<czajkowski> sinzui: what should I do about https://answers.launchpad.net/launchpad/+question/201362
<sinzui> ha
<sinzui> you saw it too
<sinzui> I just fixed it. https://code.launchpad.net/~vcs-imports/amanda/trunk
<sinzui> I types the URL wrong
<sinzui> I typed "typed" wrong too
<czajkowski> sinzui: I do tend to read all your mails :)
<czajkowski> not that I don't read the tohers
<czajkowski> *others!
<sinzui> czajkowski, We can always ask a webops to make a new series called trunk and link the new branch to it.
<sinzui> maybe we should ask the webops to make ~registry the maintainer so that we can fix the project up
 * sinzui think ~ registry should own it until someone with time can maintain it
<deryck> adeuring, https://plus.google.com/hangouts/_/ec0208c347513d03aa007b4b4114ae823e80b7a8?authuser=0&hl=en
<cr3> hi folks, how should I go about making a private PPA become public?
<Laney> czajkowski: Did it post? I don't see it on https://lists.launchpad.net/launchpad-dev/
<czajkowski> Laney: I moderated it no sign yet.
<Laney> oh woe
<czajkowski> I did noticed a massive delay on mails yesterday 50 mins in some cases
<Laney> Date: Wed, 27 Jun 2012 14:11:52 +0100
<Laney> 1.5 hours
<mrevell> jamestunnicliffe, danilos, flacoste, czajkowski, matsubara: https://plus.google.com/hangouts/_/181ae1d3328f2435b903d2a6cdc3395f400ac837?
<mrevell> jamestunnicliffe, danilos, flacoste, czajkowski, matsubara: Actually, would Skype work for all of you?
<danilos> mrevell, heh, I got this just as I was creating another one, let me kill mine
<matsubara> mrevell, yep
<matsubara> mrevell, dude!
<flacoste> mrevell: i prefer skype
<danilos> mrevell, I am on skype now, ready to go
<mrevell> jamestunnicliffe, What's your Skype id? Feel free to PM it
<jamestunnicliffe> mrevell: Erm, I would have to install it first...
<mrevell> jamestunnicliffe, I can dial you in on a landline
<jamestunnicliffe> mrevell: Actually, if we are doing voice, I have skype on my phone.
<jamestunnicliffe> mrevell: user=dooferlad
<mrevell> jamestunnicliffe, Great, yeah voice only
<mrevell> jamestunnicliffe, Can you please allow me to see your availability? I'm matthewrevell
<rick_h_> abentley: you have a sec?
<abentley> rick_h_: Sure.
<jamestunnicliffe> mrevell: Done.
* abentley changed the topic of #launchpad-dev to:  http://dev.launchpad.net/ | On call reviewer: frankban* (rvba), abentley | Firefighting: - | Critical bugs: 3.47*10^2
<jam> abentley: so are you benchmarking the branch scanner?
<abentley> jam: No, I'm making some changes to the way branch paths are resolved.
<abentley> jam: So I'm benching how long it takes to open a branch.
<rick_h_> abentley: sorry, network issues, hangout went boom
<abentley> rick_h_: ack
<abentley> jam: Actually ran into an odd SSH connection management bug that I'll file.
<rick_h_> abentley: yea, so now the test fails and the ATTR isn't read only so going to patch that up.
<rick_h_> gotta love it
<abentley> rick_h_: I think it was only testing to see that the BugListingConfigUtil version was read-only.
<rick_h_> abentley: ah ok, and with the refactor the test itself is just not valid?
<rick_h_> since buglistingconfigutil doesn't have that attribute any longer, only the model does
<abentley> rick_h_: Yes, I think so.
<rick_h_> that makes sense
<rick_h_> ok, thanks, a delet'ing I will go
<abentley> sinzui: The video in your latest post is small, making it hard to see what's happening, and there's no full-screen option.
<sinzui> I was thinking of remaking it. That was ian's example
 * Laney gives in and joins ~launchpad-dev :P
<czajkowski> Laney: hah
<czajkowski> sinzui: Am a bit unsure as to if this is something I am to do or web ops are to do https://answers.launchpad.net/launchpad/+question/201542
<sinzui> um that is confusing
<czajkowski> ah good
<sinzui> czajkowski, https://launchpad.net/python-virtkey/+admin
<czajkowski> I like you're confused also
<sinzui> this is tricky
<sinzui> It is impossible to change the Lp Id (name) and alias at the same time
<sinzui> czajkowski, remove the alias, save
<czajkowski> done
<sinzui> admin again and change the name to the alias and save
<sinzui> admin again and change the alias to the old name
<czajkowski> and change the name to what though
<sinzui> czajkowski, virtkey
<sinzui> we want to switch the text in the two field, but Lp is going to make us submit the for 3 times to do it
<czajkowski> it;s not letting me
<czajkowski> alias python-virtkey  Name virtkey
<sinzui> right, three steps, not one
<sinzui> remove the alias
<sinzui> then change the name
<sinzui> then add the alias
<czajkowski> ahh
<czajkowski> *headdesk*
<sinzui> There is contention in the db tabled when we make the change, so the site needs to make the change is small steps
<czajkowski> sinzui: nods
<czajkowski> just complicated and not straight forward, used to doing one simple change
<czajkowski> thanks for the help
<sinzui> jcsackett, This is the bug that the feature flag works with. We want to fix enough of the problem so that we can say it is fixed and the flag is gone https://bugs.launchpad.net/launchpad/+bug/885672
<_mup_> Bug #885672: Users make bugs private because they cannot hide comments <answers> <bugs> <comments> <disclosure> <information-type> <privacy> <qa-ok> <ui> <Launchpad itself:In Progress by wallyworld> < https://launchpad.net/bugs/885672 >
<jcsackett> sinzui: right, that's the one i was mentioning wallyworld was on.
<jcsackett> sinzui: so, cleanup, the two regressions, and then simplifying the flagged code and removing the flag.
<sinzui> yes, in that order
<sinzui> jcsackett, I think you reported a bug that the comment hiding ui is intrusive. maybe you can find a way to make only appear when it is needed
<czajkowski> Laney: got the mail :)
<Laney> czajkowski: Good! So, seems like mail from non-subscribers doesn't work so well
* frankban changed the topic of #launchpad-dev to:  http://dev.launchpad.net/ | On call reviewer: abentley | Firefighting: - | Critical bugs: 3.47*10^2
<cjwatson> Hmm, buildbot failed again on the same test
<rick_h_> cjwatson: oh? is that failure a repeat?
<cjwatson> Yeah
<cjwatson> If it's a regression, it's somewhere in -r15495..15502
<cjwatson> Which is staging replication stuff, bzr API modernisation, my copy_packages DB security fix, your "Add new address" form, my removal of the "single custom upload" hack, and another buildbot fix
<cjwatson> OK, it's not quite the same, the previous failure was 13 tests later in the same class
<cjwatson> No, make that one test earlier
<cjwatson> All the previous google hits I can find for this seem to be related to bzrlib
<cjwatson> But those weren't leaking storm postgres connections
<rick_h_> abentley: if you get a chance can you peek at the JS branches I've got in the queue?
<abentley> rick_h_: sure.
<rick_h_> abentley: ty much
<jcsackett> czajkowski: for some reason these pics made me think of you. desktop material? :-P http://kottke.org/12/06/birds-with-arms
<abentley> rick_h_: "That's the bulk of the LoC diff as it moves the tests under the tests. object."?
<rick_h_> abentley: ah, because it's tests.suite vs just suite.
<rick_h_> abentley: so it made sense when I wrote it :)
<abentley> rick_h_: I get ya.
<abentley> rick_h_: I assume the rest of the changes are mechanical, so r=me.  Oh, and the copyright statement doesn't need a comma after the year.
<rick_h_> abentley: ok cool
<abentley> rick_h_: Both of your LoC rationales require a waiver from flacoste or lifeless.  Do you have one?
<abentley> rick_h_: I mean, for https://code.launchpad.net/~rharding/launchpad/bug_yui35_one/+merge/112399
<rick_h_> abentley: no, I can bring it up. Or work on the other branches that should remove lines and get me the 15 I need
<rick_h_> actually, hold off abentley. Once the branch working through buildbot lands I'll be able to use this new module to reduce lines and get this down to neutral.
<abentley> rick_h_: landing lp:~rharding/launchpad/lpnames_yui35 would give you sufficient LoC credit.
<rick_h_> I'll just make the update onto that
<rick_h_> abentley: ah ok cool yea. Actually I should write these out. I've got a series of small ups and downs as I move through these files
<abentley> rick_h_: The copyright date on test_buglisting_utils.js should be 2011-2012.
<rick_h_> abentley: ah ok, thuught I'd seen them just get replaced
<abentley> rick_h_: No, the copyright date for unmodified code need to be retained.
<rick_h_> abentley: makes sense, will udpate
<rick_h_> err update
<abentley> rick_h_: r=me with those changes
<rick_h_> abentley: ty much
<sinzui> lifeless, registry.upcoming_work_view.enabled	team:linaro
<sinzui> ^ that is the UI for the feature
<lifeless> flacoste: ^ that too (thanks for catching that sinzui !)
<czajkowski> jcsackett: love the pics!
<jcsackett> czajkowski: thought you might. :-P
<czajkowski> jcsackett: have you see this weeks dekstop :)
<jcsackett> i have not. must have missed the tweet.
<czajkowski> jcsackett: https://plus.google.com/photos/102921374554385564572/albums/5730819334465556225/5757878649532660674
<jcsackett> that is an unhappy bird.
<czajkowski> jcsackett: it's a monday desktop :)
<jcsackett> mondays do tend to provoke that reaction.
<czajkowski> indeed
<czajkowski> jcsackett: it's been a fun thing to do on a monday though, leads to some very odd reactions
<jcsackett> really? i wouldn't think people would respond too oddly.
<czajkowski> yeah G+ people get rather vocal, twitter folks have fun and interact back
<jam> abentley: about bug #1018477, it may be the forking-service interfering with things.
<_mup_> Bug #1018477: repeatedly opening bzr+ssh connections to LP instances hangs <Bazaar:New> < https://launchpad.net/bugs/1018477 >
<jam> which would be really useful in debugging why it wasn't successful deploying it to production, as I haven't seen that failure before.
<jcsackett> czajkowski: huh. g+ seems to largely be filled with passive observers from my side.
<jcsackett> of course, as a passive observer, my viewpoint might be slightly skewed.
<abentley> jam: Oh, we're not using forking-service in production, but are using it on qa-staging?
<jam> abentley: everywhere but production, AIUI
<czajkowski> jcsackett: ah I do try and interact a bit on there, very useful for photography which I'm trying to learn more about
<abentley> jam: It seems plausible.  Do I recall correctly there's a way of running codehosting locally without the forking service?
<jam> abentley: you could try just flipping the config switch
<jam> use_forking_server IIRC
 * sinzui bangs head on desk
<jam> abentley: http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/view/head:/configs/development/launchpad-lazr.conf#L50
<jam> use_forking_daemon: False there
 * czajkowski passes the bottle of wine to sinzui 
<abentley> jam: Yes, I found it, and setting it False fixes the bug.
<sinzui> As much as I want to add bug tag completion to advanced bug search, this is hard to do because of the very old bug that that hand craft page is invalid because of duplicate ids
<rick_h_> sinzui: ugh, dupe ids ftl
<jam> abentley: sorry to have caused the problem, but thank you for giving some insight into why it was failing in production.
<rick_h_> sinzui: and can't add in a spare css class to hook into easy enough?
<sinzui> rick_h_, I think I can make a tighter selector based on several attrs
<rick_h_> ugh, hate attrs even more but whatever works I guess
<abentley>  jam: I suspect this should be treated as two bugs: 1. bzr fails to terminate ssh, 2. launchpad does something wrong when using forking daemon.
<jam> I know back in the day I was doing load testing, and wasn't seeing this, but if you can reproduce it reliably, I must have been doing something different.
<abentley> jam: yes, it's been consistent.
<abentley> jam: I'll bet you were using possible_transports.
<jam> it is possible that the 1-process thing is important. I think I was load testing with N workers
<jam> abentley: no, I think I just was doing 1 connection per process
<abentley> jam: That would do it too.  I was able to work around the problem by forking before branching.
<jam> for n in `seq 100`; do bzr info $branch &; done
<abentley> jam: s/branching/opening the branch/
<jam> abentley: how do you create a real branch to test this? or are you just using a non-existant branch?
<jam> (name16 doesn't  seem to have a password, and I can't create a user via the test-openid provider)
<abentley> jam: I'm using a non-existant branch.
<abentley> jam: I typically push branches up as ~mark, which is specifically configured in my .ssh/config.
<abentley> jam: I believe there's a script to create new users.
<abentley> jam: I don't think password authentication is supported.  It should be possible to add an ssh key for name16.
<jam> abentley: right, I should poke at that. In the mean-time, your script passes for me... :(
<jam> even bumping it up to 10
<jam> though it pauses for a bit at 3
<abentley> jam: What platform?
<jam> abentley: Precise running in a VM
<cody-somerville> um
<cody-somerville> When did bug titles get a max length of 66 characters?
<abentley> jam: I'm on Precise on bare metal (BM?)
<jam> I can see if I can reproduce on bare metal, but yeah so far no love.
<cody-somerville> weird. nvm. refreshing fixed it
<jam> abentley: more than one cpu?
<jam> (may or may not matter)
<abentley> jam: 4 cores, pretending to be 8.
<abentley> jam: I saw the same behaviour on devpad with qastaging.
<jam> abentley: bzr version?
<abentley> jam: bzr.dev and 2.6.0dev2 locally
<jam> abentley: interestingly, on devpad going to qastaging, the loop itself doesn't hang, but when I go to exit the python process, it hangs (waiting for ssh, I assume)
<jam> I do see 3 ssh processes just sitting around
<jam> abentley: killing one of the SSH processes with -SIGTERM was enough to get everything to clean up. weird
<jam> anyway, way past my day, but something for me to poke at
<abentley> jam: I've also seen SSH processes hanging around.
<czajkowski> sinzui: is cr3 suggestion possible on this, it;s a long way of doing it but the only way I can see  https://answers.launchpad.net/launchpad/+question/201630
<sinzui> czajkowski, you are correct. cr3 can create a new ppa, copy the packages over, then delete the ppa
<czajkowski> sinzui: grand long winded but a way to do it
<czajkowski> thanks
<cr3> czajkowski: thanks for looking into this, can you let me know the recommended way to either copy and/or move packages from one ppa to another?
<cjwatson> cody-somerville: Launchpad, sponsored by Twitter
<czajkowski> cr3: dont know if there is a recommended way of doing it.
<cody-somerville> :-]
<cr3> czajkowski: I could settle for any way :)
<czajkowski> cr3: I suspect ask in here and one of the LP folks will answer
<czajkowski> abentley: you about
<cr3> czajkowski: will do, I need to jet but I'll ask tomorrow. thanks!
<czajkowski> cr3: ok
<sinzui> jcsackett, database/schema/security.py -d launchpad_ftest_template
<wgrant> omg
<wgrant> buildbot passed
<cjwatson> phew
<wgrant> QA party time
 * cjwatson upgrades mawson again for the cause
 * wgrant defers the bzr stuff until after breakfast.
<wgrant> Still 17 minutes from buildbot completion to qa-tagging :(
<cjwatson> wgrant: Don't suppose you ever got any further with queue-api, BTW?
<cjwatson> I know it's a bit of a bear of a branch
#launchpad-dev 2012-06-28
<StevenK> cjwatson: You're dealing with the QA of r15504?
<StevenK> jelmer has a bunch of QA. :-(
<cjwatson> Yes, literally just finished it in fact
<StevenK> cjwatson: \o/
<StevenK> wgrant: branch-unsubscribe-revokes is looking good so far, ~ 2 hours to go
 * StevenK gets disgusted enough to run format-imports again.
<StevenK> cjwatson: Can we delete delayed copies yet? How about now? Now? :-P
<wgrant> Once the custom upload fix is done and pocket queue admin permissions work :)
<wgrant> I think those are the only two blockers.
 * StevenK stabs the branch scanner.
<StevenK> Every time I get frustrated, ssh to carob and sync logs it then scans correctly.
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/imports-really-this-time/+merge/112470
<wgrant> StevenK: lib/lp/app/widgets/tests/test_popup.py is wrong
<wgrant> And wow I completely failed at bugsummaryrebuild
<StevenK> Oh, yeah, I completly missed the utter fail in test_popup
<StevenK> Let me fix that up
<StevenK> wgrant: Diff updated
<wgrant> r=me
<wgrant> Your nation thanks you for your service.
<StevenK> wgrant: You have an hour to object to branch-unsubscribe-revokes.
<wgrant> StevenK: doit
<StevenK> wgrant: And the garbo job? :-)
<wgrant> Objection
<StevenK> wgrant: And the discussion about ubuntu/precise/datereleased over our API in -devel?
<StevenK> wgrant: Can has a concrete ruling on the garbo job, rather than 'Objection' ?
<wgrant> StevenK: It can land once the unsubscribe stuff is deployed and we've performed some data checks.
 * StevenK stabs lazr.enum and then twists the knife.
<StevenK> wgrant: So, I've been fighting with a generalisation branch for RBSJ for too long.
<StevenK> I think we need a seperate job
<StevenK> As much as pains to say
<wgrant> StevenK: Why?
<StevenK> wgrant: Because I'm getting damn well nowhere
<StevenK> Or it feels like it
<wgrant> What's the problem? :)
<StevenK> Perhaps starting with renaming every class I could find was not the best way to start.
<wgrant> Sounds fine to me.
<StevenK> wgrant: http://pastebin.ubuntu.com/1063704/
<wgrant> StevenK: Looks not unreasonable. Is there a problem?
<StevenK> I was chasing through the importfacist again
<StevenK> Turns out lazr.enum gives unintelligble errors if the first line of a DBEnum is multi-line.
<jtv> Hi lifeless â are you free for a question about job dispatch with celery/rabbit?
<lifeless> jtv: high latency but yes. Just ask
<jtv> We want to dispatch jobs to workers through celery.  But we'd like to get proper transactionality on that.  We looked at django-celery-transaction but it's written from a mubbet-labs âjust patch into commit/abort and do your work thereâ approach.
<jtv> As opposed to the âwhatever is in the database is the valid stateâ approach that you'd normally go for.
<jtv> We'd like to avoid rolling our own code for transferring committed jobs from the database to rabbitâ¦ know of any ready-made alternatives?
<jtv> There's django-async, which at least is written by someone who understands this stuff, but it was designed as a lightweight replacement for celery rather than as complementary to it.
<lifeless> no; in particular you'll need to have something poll, or something using postgresql's push facilities
<lifeless> I thought the plan was to avoid the need for that by not emitting state; just emitting 'something happened' events and letting the worker read-back ?
<lifeless> so you will be transactional anyway
<jtv> Well, sort of.
<jtv> Yes, the plan was to do that.  I hadn't thought of it as a replacement for transactionality of messaging; we'd always have to call back to the central server even if just to see whether the requesting transaction had committed.
<jtv> And hope that we'd not do so too fast for the commit to happen.  :)
<lifeless> well, yes, unles you emit the event right after
<lifeless> which may lead to no notification, of course
<lifeless> jtv: things to watch out for: super big journal tables after doing this for a long time
<jtv> Journals on the filesystem?  Or do you mean un-vacuumed database tables?
<lifeless> unpruned specifically
<lifeless> you're basically building a queue in the db, be sure it gets cleaned out
<jtv> So you are talking about tables in the database then?
<lifeless> yes
<jtv> Well there's vacuum for those.  Right now I'm not building a queue in the db; I'm asking if there's a good way to do it.
<lifeless> I mean lots of rows, not unvaccuumed pages
<lifeless> other implementations I've seen accumulate forever, failing to delete rows.
<jtv> That seems silly.
<lifeless> yes ;)
<jtv> So basically we don't know of anybody getting this right?
<jtv> At least, with celery.
<lifeless> U1 are doing a roll-your-own implementation
<lifeless> for their materialised views implementation
<lifeless> the only precanned 'this is a robust answer' I know of our there is storm.
<lifeless> Which is probably not something you'd want for MAAS.
<jtv> No.  I didn't even know it integrated with rabbit.
<lifeless> it doesn't particularly but it knows how to deal with the general situation
<lifeless> jtv: it seems to me you're solving a single piece of the puzzle better than the rest of the stack supports it
<lifeless> jtv: amqp isn't intrinsically transactional
<jtv> Sadly.
<lifeless> unless you run in specific 2pc mode
<lifeless> erm, specific trasnaction mode
<lifeless> this means, you may suffer dropped messages
<lifeless> if rabbit is shut down hard at an untimely point.
<lifeless> This means, that its no worse to just emit the celery event right after the db commit.
<lifeless> Its not a new failure mode.
<jtv> That is sad.
<lifeless> whatever approach is used to deal with 'some numpty/event killed rabbit hard' will also deal with 'the appserver was killed/segfaulted between the db committing and it handing off the event to rabbit'
<lifeless> I may be wrong; critical analysis welcomed.
<lifeless> I need to run for a minute, baby->sleep.
<jtv> I'll be here.
<lifeless> back, tho my focus won't be here
<lifeless> its mid-evening now
<lifeless> I'll keep replying if you want to chew on this more
<lifeless> just slower - sorry.
<mgz> anyone know where the source for the launchpad qa bot is?
<nigelb> pqm?
<nigelb> https://launchpad.net/pqm
<wgrant> mgz: That's qa-tagger
<lifeless> not opened sadly
<wgrant> Creatively enough that's lp:qa-tagger
<lifeless> needs personal details purged to do so
<nigelb> heh
<wgrant> Oh, I thought that was free. Sad.
<mgz> ta!
<jtv> lifeless: some distractions on this end as well.  Anyway, I was also somewhat concerned about immediate failures to handle a job request.  It would seem more sensible if those could abort the transaction that requested them.
<jml> oh
<jml> also, james_w made something that's a bit like qa-tagger, except it tracks revisions rather than bugs.
<jml> unfortunately, I can't remember what it's called or where it lives.
<jml> one of the downsides of ec2/canonistack deployment is the lack of memorable hostnames
<jtv> jml: the openbsd folks had a solution for that â use pokemon names.  But I suspect even that is not a large enough set for EC2.
* jelmer changed the topic of #launchpad-dev to:  http://dev.launchpad.net/ | On call reviewer: jelmer | Firefighting: - | Critical bugs: 3.47*10^2
<jelmer> jtv: /memorable/ hostnames ? :P
<jtv> jelmer: well I remember pikachu, and that's about the only pokemon I ever heard of.  So it seems to be working.
<jelmer> that's also as far as I get..
<jml> jtv: also, not memorable. as a case in point, my code is running in production on machine named after an obscure fruit.
 * jelmer kind of likes the fact that canonical machine names make me learn more obscure English words
<jtv> Come on, there has to be a way to re-pair my laptop to my bluetooth keyboard!
 * jml just had an idea
<jtv> jml: well, what's the idea?
<jml> jtv: it's pretty revolutionary
<jml> (and magical)
<jtv> Come on, cut to the chase!
<jml> a table: Column 1 is 'machine name', Column 2 is 'list of services on that machine, in names that developers use to refer to them'
<jml> one of those per project
<jml> and then there's none of this "could you please run X on foo-svc?", "sure. is that snuffleberry?", "I don't know, you deployed it."
<jtv> That would be nice, yesâ¦ we had something like that on the wiki, once upon a time, but I don't think it was maintained very actively.
<jtv> I hate that situation.
<jml> me too.
<jml> I also hope that there isn't actually a machine called snuffleberry.
<jml> I've seen a couple of diagrams, and they do the job too.
<jml> but they are a little trickier to maintain & grep.
<lifeless> jtv: you mean if celery fails to dispatch ?
<jtv> lifeless: yes
<jtv> For example, because the data won't serialize.
<lifeless> jmm
<lifeless> so, I'd have thought the django glue would do that immediately, and only defer the dispatch
<jtv> That would be nice, if possible.
<jtv> Maybe it does.
<wgrant> jml: But then you have the problem of services that have like 5 different names depending on who you ask :)
<wgrant> (carob:~stevenk/name-to-service is pretty good for LP, though)
<jml> wgrant: well yeah, but one problem at a time.
<StevenK> I didn't know anyone actually used it
 * nigelb notices the Critical bug count being less than 3.5 * 10^2.
<wgrant> nigelb: That's a lie.
<wgrant> There's 402 open critical tasks
* wgrant changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: jelmer | Firefighting: - | Critical bugs: 4.0*10^2
<nigelb> :(
<nigelb> I should have kept quiet and lived the lie
<czajkowski> nigelb: dont comment :)
<cjwatson> StevenK: I actually started on a remove-delayed-copies branch last night, although wgrant's right that pocket queue admin needs to be done first.  Other things that come to mind are converting the +copy-packages UI to PCJs (can it perhaps use PCJs for everything now rather than having the dodgy "am I going to be able to copy synchronously" check?  is it possible to update the UI on the progress of a job?), and making ...
<cjwatson> ... Archive.syncSource start failing when it's asked to do a private->public copy that would require a delayed copy.
<cjwatson> (Or perhaps convert Archive.syncSource to be asynchronous in that case, although that kind of violates its advertised contract.  Might be better to fail and get people to convert to Archive.copyPackage.)
<cjwatson> StevenK: Also I know that some bits of PCJs are hidden behind feature flags or disabled and I'd like to understand why.
<mgz> jam: http://pastebin.ubuntu.com/1064002/
<cjwatson> And of course go through all the delayed copy tests and work out which of them might be sensibly converted to apply to direct-copies/PCJs.
<cjwatson> Ah, the soyuz.copypackageppa.enabled feature flag is due to having no way to give feedback.  Anyone want to educate me on how that stuff might work?
<cjwatson> bigjools: In bug 575450, you offered mentoring for anyone who wanted to sort out feedback in PPA pages from the new async copying code.  I'm game, since this seems to get in the way of cleanup I've committed to do as part of feature work I've already done.  Do you think it could work in the same way as feedback from distroseries initialisation?
<_mup_> Bug #575450: Archive:+copy-packages nearly unusable due to timeouts <lp-soyuz> <ppa> <qa-ok> <timeout> <Launchpad itself:In Progress by rvb> < https://launchpad.net/bugs/575450 >
<StevenK> cjwatson: I'd prefer Archive.syncSource to fail, TBH
<cjwatson> StevenK: OK.  That obviously is definitely blocked on pocket queue admin, since we need things to be self-service for the security team before breaking what they're using now.
<bigjools> cjwatson: I am a little tired right now, just had twins, so if you can wait a couple of weeks I'll be happy to help
<cjwatson> bigjools: hah, congratulations!
<bigjools> ta :)
<cjwatson> go change one nappy with each hand or something then
<StevenK> cjwatson: If the distroseries init stuff is making use of IDSJ, it can probably be used as inspiration
<cjwatson> StevenK: It is
 * StevenK heads off to pick up his wife
<cjwatson> There's a thing where the job calls notifyUserError on failure and that pops up through distroseries-portlet-derivation.pt
<cjwatson> So I can probably monkey-see-monkey-do that if it's broadly a usable approach
<bigjools> cjwatson: the plan was to have failed jobs showing on the page and to have someone ack them by deleting them
<bigjools> anyway, cheerio
<cjwatson> Hm, right, a bit different then
<cjwatson> r14665 seems to be a moderate-sized chunk of that
<cjwatson> rvba: Do you happen to recall what was left to do in bug 575450?  Your r14665 seems to be approximately what bigjools said above needed to be done
<_mup_> Bug #575450: Archive:+copy-packages nearly unusable due to timeouts <lp-soyuz> <ppa> <qa-ok> <timeout> <Launchpad itself:In Progress by rvb> < https://launchpad.net/bugs/575450 >
<cjwatson> I wonder if it's just cleanup at this point ...
<mgz> Mr Reviewer Please: https://code.launchpad.net/~gz/launchpad/py27_xmlrpc_transport_timeout/+merge/112528
<cjwatson> rvba: If the UI does indeed now do what bigjools said it needed to, it is extremely tempting to propose something like http://paste.ubuntu.com/1064032/ for merging
<cjwatson> (Does it matter if the DB has feature flags that are no longer referenced anywhere in code?)
<cjwatson> Hm, OK, this runs headlong into bug 795005
<_mup_> Bug #795005: Asynchronous PPA copying with default series OOPSes <oops> <ppa> <Launchpad itself:Triaged> < https://launchpad.net/bugs/795005 >
<StevenK> cjwatson: If the DB has feature flags that no longer referenced, they should be removed.
<cjwatson> StevenK: Right, but will it make anything fail if they aren't - that is, are there deployment considerations in landing patches that remove feature flags?
<StevenK> cjwatson: Right, so I've in fact just done this.
<wgrant> cjwatson: It's fine to remove the code first.
<StevenK> cjwatson: r15494 drops two related feature flags from the code.
<wgrant> cjwatson: Nothing breaks if there are unknown flags set.
<wgrant> Or unknown scopes
<wgrant> etc
<StevenK> cjwatson: I was going to ask that the two flags be removed from prod tomorrow morning.
<cjwatson> Cool.
<jam> jelmer: mr reviewer, if you please: https://code.launchpad.net/~jameinel/launchpad/py27-parse-response-1018768/+merge/112537
<lifeless> cjwatson: flags were designed to fit a relaxed workflow
<lifeless> cjwatson: vs lazr.config which is designed to fit a strict, but high latency work flow.
<czajkowski> lifeless: have you come across this issue or is there a step missing ?https://answers.launchpad.net/launchpad/+question/201678
<jelmer> czajkowski: he should add a release, that's the way to add downloads
<czajkowski> jelmer: ahh I see
<czajkowski> confusing me aslso
<czajkowski> thanks
<cjwatson> Sounds like an opportunity to improve the docs then :-)
<jelmer> czajkowski: I've replied on the question page
<czajkowski> jelmer: ack, I can view his screen shot so can see his point
<czajkowski> cjwatson: indeed
<cjwatson> rvba: Also, is it possible that your r14665 fixed bug 812869?  It certainly looks as though it should have done
<_mup_> Bug #812869: Failed PackageCopyJobs should show up on the PPA page somehow <derivation> <ppa> <Launchpad itself:Triaged> < https://launchpad.net/bugs/812869 >
<rvba> cjwatson: let me refresh my memory and have a look at the revision and the bug in question :)
<rvba> cjwatson: that would make sense indeed.  Also, what I've done in my branch is exactly what Julian suggests to do in the description of the bug, for that type of error that is (if the target archive is a ppa).
<cjwatson> Yeah, that's what I thought
<cjwatson> Difficult to easily try it at the moment since async copies are disabled
<cjwatson> Speaking of which:
<cjwatson> jelmer: Oh hi Mr. OCR.  Fancy having a look at https://code.launchpad.net/~cjwatson/launchpad/copy-packages-with-default-series/+merge/112557 ?
<jelmer> cjwatson: yep, I'll have a look
<cjwatson> I *think* that should let us re-enable async copies in the UI.
<cjwatson> Assuming there are no other lurking undocumented reasons.
<deryck> Morning, everyone.
<rick_h_> morning
<jam> cjwatson: I'd like to have a chat with you sometime about how we can get some of the 'when is it safe to cleanup files' policies encoded into Launchpad, rather than having them be as ad-hoc as they are now.
<cjwatson> Mm.  Well, at the very least it seems as though it ought to be a script in the tree that web0ps can run, rather than it being ad-hoc pastebins.
<deryck> adeuring, https://plus.google.com/hangouts/_/f9d8daf3da1ea6e3b6c53ff8ed10332dbde5899c?authuser=0&hl=en
<cjwatson> Unfortunately the fact that PES have a habit of relying on obsolete distroseries after Ubuntu stops supporting them, and we have to confirm with them that they no longer care, makes it difficult to do it all automatically.
<jml> I just hit a timeout loading my branch page :(
<jam> cjwatson: PES?
<jam> also, we can make it manual as much as "we've confirmed it is ok to nuke N, do it"
<cjwatson> The division formerly known as OEM
<jam> so there is a bit of manual step in there, but all of the validation that we're doing is mostly already done inside launchpad, we just have to realize if we can trust it.
<mgz> lib/lp/services/doc/timeout.txt seems appropriately named
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: jelmer,jcsackett | Firefighting: - | Critical bugs: 4.0*10^2
<rick_h_> jcsackett: feel like some JS in the morning? https://code.launchpad.net/~rharding/launchpad/bug_yui35_two
<jcsackett> rick_h_: sure.
<jcsackett> rick_h_: it's only because i want these changes landed that i'm not going to yell about diff size. :-P
<rick_h_> jcsackett: well, don't let it fool you, it's 98% "tab over all the linse"
<jcsackett> rick_h_: yeah, i'm seeing that. i'm just giving you a hard time, anyway. :-P
<rick_h_> :) I'll send the 'gift beverage' later
<rick_h_> hopefully I'll have one more of these before EOD
 * jcsackett laughs at "gift beverage".
<jcsackett> i wish diffs were smarter about tabovers.
<jcsackett> rick_h_: how many tests have you seen in YUI land that rely on .wait() in some form?
<jcsackett> i'm vaguely concerned about those in relation to gary_poster et al's info regarding tests with programmed delays being problematic for parallel testing.
<rick_h_> jcsackett: in our code?
<jcsackett> rick_h_: yeah.
 * gary_poster on call
<rick_h_> there's a solid handful I think, but I think most of that stuff has started to get 'skipanimation' flags and such to help clear that up
<jcsackett> rick_h_: okay, cool.
<rick_h_> I've not gone through everything though so can't say for sure
<jelmer> cjwatson: check_copy_permissions properly checks with the source distroseries of each spph if none is specified explicitly; however, copy_asynchronously uses the dest_series of the first specified spph for all specified spphs if no distro_series was explicitly specified.
<jelmer> cjwatson: apologies for the abundant use of the word specified in that sentence..
<ivory> jelmer: could you take a look at this please? https://code.launchpad.net/~ivo-kracht/launchpad/bug-436663/+merge/112576
<jelmer> ivory: yep
<ivory> jelmer: thanks
<vila> gmb: for your pleasure https://code.launchpad.net/~vila/launchpad/py27-mail-header-continuation-lines/+merge/112579
<sinzui> jcsackett, do you have time to review https://code.launchpad.net/~sinzui/launchpad/bug-tag-completions/+merge/112580
<jcsackett> sinzui: sure.
<cjwatson> jelmer: Oh, yes, I see what you mean (after some thought)
<cjwatson> jelmer: Fixing
<gary_poster> jcsackett, I see I was mentioned in passing.  Thank you for keeping that in mind!  The biggest thing is to make sure that you allow for a testing machine being much, much slower than you expect.  We don't want the tests to hang forever, but if it passes in a really long time because of contention on [X random thing I wish I knew but is not apparently RAM, CPU or I/O] then we'd rather give it a nice long while to try
<gary_poster>  and let it pass.
<cjwatson> jelmer: OK, fixed now
<cjwatson> Er, modulo pushing (pushed now too)
<cjwatson> StevenK: A bit to go, but very provisionally the gain from removing delayed copies looks like 1160 LoC
<ivory> sinzui: jelmer said a ui reviewer should take a look at this and since you are the only one, could you? https://code.launchpad.net/~ivo-kracht/launchpad/bug-436663/+merge/112576
<ivory> sinzui: i attached a screenshot to the bug description
<rick_h_> sinzui: so you can set a CSS name override on extending a Widget: http://yuilibrary.com/yui/docs/api/files/widget_js_Widget.js.html#l95
 * sinzui looks
<rick_h_> sinzui: not sure if that helps you with the issue in your MP addendum
<sinzui> rick_h_, !
<sinzui> rick_h_, I bet it does.
<cjwatson> ivory: Fixing that bit of lint would just amount to deleting one line; I'm not a reviewer but IMO you might as well do that
 * sinzui wonders how quickly he ca rewite the widget again
<rick_h_> sinzui: heh, you want me to look over the rest here?
<sinzui> rick_h_, please do.
<sinzui> ivory, that is a good improvement.
<ivory> sinzui: i was a little bit concerned about the little gap next to the bug icon but i have no idea how to fix that
<sinzui> ivory, one of the links is not using a sprite properly
 * sinzui looks at code
<sinzui> ivory, this looks like one of the macros is bad, but fix it would require a review of many pages that use the macro
<sinzui> ivory, few mp reference both blueprint and bug, so the issue will rarely be visible
<ivory> sinzui: so its ok for now?
<sinzui> yes
<ivory> ok then thank you for helping me
<jcsackett> sinzui: r=me, but there's a short style note on the MP.
<sinzui> thanks
<jcsackett> yw.
<cjwatson> jelmer: Thanks for the review.  I thought that test_copyPackages_with_default_distroseries was exactly such a test, or am I misunderstanding you?
<cjwatson> It creates two SPPHs with different distroseries, and copies both with to_series=None.
<jam> cjwatson: we're trying to sort out https://answers.launchpad.net/launchpad/+question/194817
<jam> I think we've unblocked ubuntu translations to open Q for translations, but it is pending DPM actually exposing Q to users
<jam> (specifically, unchecking the Hide translations for this release box)
<jam> I haven't seen dpm around in about a week, so I don't know if there is something blocking. Do you know anything about it?
<jam> jtv said that the import queue looked good
<cjwatson> jam: Right, thanks.  From my point of view the question is a fire-and-forget notification which I am required to emit as part of NewReleaseCycleProcess but otherwise don't care much about. :-)
<cjwatson> I don't know anything about dpm's status on it.
<jam> czajkowski: then I think we can close the question
<jam> as we took his info, and did our part of it
<james_w> dpm is on vacation
<rick_h_> sinzui: couple other suggestions but good stuff!
<jelmer> cjwatson: ah, that's true..
<sinzui> thank you
<jelmer> cjwatson: did that test fail before r15510 ?
<james_w> he's back in 7 days
<czajkowski> jam: grand will do thanks
<rick_h_> jcsackett: heads up, YUI3.5 doesn't like the selector stuff [id=field.name], it has to be in quotes in 3.5 [id="field.name"]
<jcsackett> rick_h_: i was gathering that from your updates.
<rick_h_> jcsackett: so any time we can catch that now the better
<cjwatson> jelmer: No - that comes under my comment in the MP about not adding a browser test because there's one that fails already once I remove copy_synchronously
<cjwatson> jelmer: I can try to figure out if *that* test would have failed before r15510
<rick_h_> jcsackett: yea, I've got a google doc with the new things to watch out for. Going to get a bit farther before I wiki/send that out to the list
<cjwatson> (Too many branches!)
<cjwatson> And for that matter I ought to test that the failing test passes with r15510
<jcsackett> gary_poster: ok, i think most of our tests in YUI have a fair bit more padding than they need, so perhaps they'll be alright. and as rick_h_ has mentioned, we're slowly removing a lot of the need for delays anyway.
<gary_poster> awesome
<cjwatson> Hm, it doesn't check for the bug fixed in r15510
<cjwatson> jelmer: OK, let me stop trying to dodge it and see if I can figure out how to add a proper browser test or two here
<jcsackett> rick_h_: ah, i see your comments on sinzui's diff. now i understand you're highlighting it. good catch, and thanks.
<rick_h_> jcsackett: yea, just pushing the word out since you're reviewing/doing JS stuff. I know I hit up a review for ian bringing it up as well
<jcsackett> rick_h_: yeah. there's a slight mental lag in realizing something is needed and applying that to reviews. :-P
<jelmer> cjwatson: okay, great
<cjwatson> jelmer: Perhaps I should convert at least some of archive-views.txt to unit tests while I'm at it ...
<cjwatson> I'm rarely confident that I can extend doctests without either (a) breaking something or (b) wanting to shoot myself to ease the pain
<jelmer> cjwatson: You should get some kind of public service medal if you did that.. :)
<cjwatson> I assure you it's entirely selfish
<jelmer> cjwatson: I'm mostly concerned about corner case bugs, so as long as the test coverage is decent I'm happy
<cjwatson> Yeah, me too
<cjwatson> Since I'm trying to accelerate making the async copy stuff the only path
<cjwatson> One of those things in Launchpad where it looks like it's been about 90% done
<sinzui> jcsackett, I will make the changes you ask, but I am not being pythonic. That function is an method argument ending in ");"
<sinzui> jcsackett, it would be more obvious if I defined the function first, then passed it to the method
<jcsackett> sinzui: well, i see that one is a creation call, one is an event call binding, and one is a function def. my understanding was that for all of those closing at the same tab level as the start, rather than the enclosed, was preferred.
<jml> wtf
<jml> another timeout
<jcsackett> sinzui: if i'm wrong as goes JS style (or LP's version of it) i'm happy to be told so. :-)
<sinzui> jcsackett, js hackers have inconsistent style because JS does not require it, and encourages anonymous inline definition to manage scoping. I am very consistent since I know that how I wrote my method calls will not change if I choose to no use anonymous functions
<jcsackett> sinzui: fair. we should probably at some point then figure out a rule for consistency's sake in our code. but it's not terribly important if you're aginst it -- rick_h_'s points are definitely more key.
<rick_h_> sinzui: yea, I think in JS you always tend to have at least one closing }; lined up with the start of a block
<rick_h_> ime that is
<rick_h_> but you're right that it's more of a practice thing than something jslint, etc will yell about
<rick_h_> ummm, jcsackett did I ever tell you that you're a great co-worker? https://code.launchpad.net/~rharding/launchpad/bug_yui35_three/+merge/112590
 * rick_h_ runs to go fetch some lunch far far away
<jcsackett> right, everyone else? rick_h_ is why your MP is not being looked at today. blame him.
 * jcsackett settles down to a long review.
<rick_h_> jcsackett: it was only 3 files? not my fault there are nearly 4k long JS files in the tree
<rick_h_> jcsackett: these are even fewer changes than the earlier set
<jcsackett> rick_h_: yeah, it doesn't actually look that bad.
<jcsackett> but diff size: 7000 has a way of setting one back on one's heels. :-P
<rick_h_> yea :/
<rick_h_> makes you feel better, I wasn't going to do a _four, but kind of have to
<jcsackett> eh, i don't really need to be made to feel better. the only real problem with these branches is that the diff ordering makes it impossible to tell what's just indenting and what's an actual new line.
<rick_h_> only new lines shold be the top of the files
<jcsackett> rick_h_: ah, that makes things easier, then.
<rick_h_> I'm really only changing the top 20 lines of files, and indenting the rest. Oh and each suite.add is now tests.suite.add
<rick_h_> in this one I had the s/LPClient/Y.lp.testing.helpers.LPClient to fix that bug/code dupe
<rick_h_> I'll try to note explicitly in the next one.
<czajkowski> 8/c
<mgz> jam: bug 1018905 filed
<_mup_> Bug #1018905: lp/services/doc/timeout.txt hangs in Python 2.7 <python-upgrade> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1018905 >
<gmb> vila, http://paste.ubuntu.com/1064552/. The failure is somewhere before line 2664
<rick_h_> sinzui: can you hit up #315 of the updated MP ?
<rick_h_> sinzui: sorry, the id=field quotes
<sinzui> ruse
<sinzui> sure
<gmb> gary_poster, jelmer just found our problem: lp.codehosting.codeimport.tests.test_worker.TestBzrImport.test_unsupported_feature  starts but never (apparently ) finishes.
<gmb> Keeping that test: line gives you 17 tests (not parsing out all the crud we parsed out). Removing it gives you 800-something.
<gmb> Will poke at it properly tomorrow.
<gmb> gary_poster, http://paste.ubuntu.com/1064576/ is the subunit stream. The test is at line 70.
<jelmer> gmb: bug 507804
<_mup_> Bug #507804: damaged streams may not be obvious <subunit:Triaged> < https://launchpad.net/bugs/507804 >
<cjwatson> wgrant: I believe I have fixed https://code.launchpad.net/~cjwatson/launchpad/custom-uefi/+merge/111626 in light of your configuration comment, by handling this in publisher configuration as well.  Do you think this is a better approach now?
<gmb> gary_poster, See that bug that jelmer just pasted ^^
<gmb> Fun times.
<gary_poster> gmb :-) .  I don't see how that subunit bug will be fixed, exactly.  makes the whole thing fragile, but...not sure what else subunit could do other than various heuristics.  Meanwhile, we should fix that test, and then hopefully we'll be ok.  There may be other tests like that though--note that other workers seemed to be generating the "unknown worker" tests too.
<cjwatson> jelmer: I've added that test now (though I gave up on the unit test conversion, sorry :-)  I found some relevant tests that were already unit tests instead) and will land that shortly.  Thanks for the feedback.
<cjwatson> jelmer: (And the new test failed before r15510)
<jml> did anyone here just expire launchpadlib 1.6.0 on pypi?
<jml> cause now I can't 'pip install launchpadlib==1.6.0'
<jml> which means I can't have a virtualenv that reliably looks like lucid
<rick_h_> jml: yea, and not seeing it on crate.io either.
<rick_h_> jcsackett: last one https://code.launchpad.net/~rharding/launchpad/bug_yui35_four/+merge/112609
<jcsackett> rick_h_: on it.
<jml> also oauth is missing
<jcsackett> rick_h_: r=me.
<rick_h_> jcsackett: ty much.
<sinzui> deryck, rick_h_Do either of you have any insight into why I cannot between the items in a choicelist? For example I open bug status, the close button is focused. I cannot tab to the items in the list...though I know all but one is a anchor.
<rick_h_> sinzui: looking
<rick_h_> sinzui: sinzui can you tab outside of a form?
<sinzui> well...
<rick_h_> I'm trying to find something that confirms/denys this but no luck yet
<deryck> sinzui, I recall some bug about this before too.  Seems like we used tabindex to fix it.
<rick_h_> right, the tabindex docs don't state if they only work inside of forms or not
<sinzui> rick_h_, no but I just hacked the picker to add tabindexes to the anchors tabbing from the close button goes to the top of the page (leaving the overlay) then eventually returns to 2 of the link many links listed
<rick_h_> http://www.w3.org/TR/WCAG20-TECHS/H4
<sinzui> this is madening
<rick_h_> sinzui: right, but what's the order on the tabindex?
<rick_h_> it works on dom order by default, so top of the page makes sense
<sinzui> 0 for the button, 1-n for the items in the list
<rick_h_> try setting 0 on the close button and then 1+ for the links
<sinzui> sorry, isn't that what I did?
<sinzui> ah The links to have the tabindexes, but the button did not stick.
 * sinzui looks
<rick_h_> sorry, when you said 0 for the button didn't follow
<sinzui> rick_h_, I don't appear to be setting tabindex="0" on the close button. There is no error, maybe I added it to some other overlay's button. The link do have a sequence on tabindexes
<rick_h_> sinzui: yea, and tested that it works sans form no problem
<rick_h_> so must be a matter of getting the order/relationship going with the rest of the page noise
<sinzui> Maybe I should start the index at 100,000
<rick_h_> hah, there you go
<rick_h_> so is there some code in the widget setting focus on the close icon?
<rick_h_> ah yea, there it is
<sinzui> yes, after we render we position and set focus...
<sinzui> but that is also were I decided to add the tabindex and it does not show
<rick_h_> using set or setAttribute?
<sinzui> I think I need to step through the code to see if I enter that method
<rick_h_> try setAttribute vs just set('tabIndex')
<rick_h_> but yea, debugger to make sure you're hitting. If the close button has focus though I'd assume it must get hit
<sinzui> ah
<rick_h_> so as a user, I'd think we'd focus on the first choice, and then end on the close button
<sinzui> damn it
<sinzui> I am a moron
<sinzui> I amde the same mistake last week
<rick_h_> heh, if you're a moron I give up man
<rick_h_> man, maybe I'll tinker with this widget. Would be so nice if it supported esc for close, tab through, enter for selection
<rick_h_> totally should be doable looking at the code
<sinzui> rick_h_, that is exactly what I want to solve
<rick_h_> ah, very cool
<sinzui> I cannot use keyboard to report a bug anymore because the new widgets are missing keyboard support
<rick_h_> so yea, what I'd do is focus on the first selection item, set tabIndex on all of them, and add a keyUp listener to the bindEvents
<rick_h_> handling close, select, etc via the event keycode
<sinzui> yep
<sinzui> there is another bug some where about subscriber overlays not dismissing on esc. I was going to look at that too
<rick_h_> I've got to run, but let me know if setting a large tabindex with setAttribute works out. I'm curious if hte browser picks up on dynamic index settings
<lifeless> czajkowski: hi sorry was waaay ED
<lifeless> cjwatson: btw, user confusion -> we should fix the product, *then* fix docs ;) - ideally. but we may have to do docs as a stopgap.
<cjwatson> lifeless: well, quite - but both of those before answering the question but doing neither
<cjwatson> (which is a tremendously easy thing to fall into, guilty as charged)
<lifeless> cjwatson: +1 :)
<lifeless> james_w: did that pypi weirdness get resolved ?
<james_w`> lifeless, somewhat
<james_w`> lifeless, there's probably a message in the canonical-launchpad moderation queue
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
<james_w> so you guys can't see my response because I'm not subscribed to canonical-launchpad, and we can't see sinzui's response because he's not subscribed to consumer-applications, hilarity ensues
<sinzui> ha ha
<sinzui> Instead of having mailing lists, Lp, could offer conversation on demand. You start one, invite users and teams, and we let the thread end in a few days
<james_w> sinzui, was there anything in your response that can't wait until tomorrow?
<sinzui> no
<mwhudson> james_w: i can give you my notmuch invocations that make subscribing to canonical-launchpad painless :)
<james_w> heh
<james_w> sinzui, ok, thanks
<mwhudson> mostly NOTMUCH tag -inbox -unread tag:inbox from:script-failures@launchpad.net
<mwhudson> oh and something for oops summaries too
<sinzui> james_w, I suggested using the series which is often more accurate https://launchpad.net/launchpadlib/trunk
<james_w> ah, ok
<james_w> I don't know if we can do that in setup.py
 * StevenK peers at buildbot.
<spm> s/peers/stabs/
<StevenK> That would be nice.
<wgrant> cjwatson: Let me see...
#launchpad-dev 2012-06-29
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/rbsj-generalise/+merge/112684
<StevenK> wgrant: Can haz review?
<wgrant> StevenK: Looks good
<wgrant> Oh
<wgrant> It's docutils
<wgrant> That's why WADL generation is so slow.
<wgrant> It passes all the docs through docutils
 * wgrant headdesks
<StevenK> Haha
<StevenK> Trying to speed up make?
<wgrant> Yeah
<wgrant> Removing docutils speeds it up by about 80%
<wgrant> But I don't think we can just stop docutilsing :(
<StevenK> wgrant: http://pastebin.ubuntu.com/1065497/
<wgrant> StevenK: Good start, although it doesn't look liek it actually unsubs yet
<StevenK> It does not, no.
<StevenK> I need help with that bit, since RASJ.run() seems to use BTF for everything.
<wgrant> cjwatson_: I don't quite understand the reason for the divergent paths in lib/lp/soyuz/scripts/packagecopier.py. Shouldn't it just be defaulting series to the one on the SPPH, with no other differences?
<wgrant> StevenK: Right. Initially you could just loop through the branches and call Branch._checkBranchVisibleByUser. Once we redefine branch security in terms of the new access schema, we can migrate to something better.
<wgrant> Using the slow awful APIs isn't so bad for branches, since there are very few private ones.
<StevenK> wgrant: for branch in self.branches: for sub in branch.subscribers: branch._checkBranchVisibleByUser(sub) does not seem full of win and puppies.
<StevenK> sub.person, but details
<wgrant> Hmm
<wgrant> So
<wgrant> We need this for revocation from +sharing
<wgrant> It is perhaps best to do this straight against the new access schema, since we'll hopefully have that populated early next week.
<wgrant> StevenK: So you'll want to prepare this branch, using something almost identical to the BugTaskFlat query, but keep it on ice until all the garbo stuff is done next week
<StevenK> wgrant: All I'm saying that is going to be slow and horrible. If it's going to be replaced after branch.access_grants is populated, then I withdraw my objection, since it should be shortlived.
<wgrant> StevenK: But the job isn't actually useful until we're using branch.access_grants, so it can just use that frokm the start
<StevenK> Right
<StevenK> So I should land the branch.aag garbo job? :-)
<wgrant> StevenK: Not until the unsubscribe fix is deployed everywhere
<StevenK> wgrant: Which I wanted to deploy this morning and you told me to wait :-P
<wgrant> StevenK: buildbot will be done in half an hour, you can deploy then :)
<wgrant> If you had explained your reasoning for wanting to deploy urgently this morning, we could have.
<StevenK> wgrant: I had forgotten the branch.unsubscribe fix was blocking the garbo job. :-(
<StevenK> Right, buildbot done
<StevenK> wgrant: Do we want to wait for Europeans to QA or polish off the QA?
<jelmer> StevenK: jam and mgz seem to be quite keen on doing their own, to see what it's like
<adeuring> good morning
<StevenK> jelmer: Feel free. I'd like to deploy at least up to r15520
<gmb> jam: http://oo00.eu/
<jam> StevenK: so we qa'd are own, but the 15518 is currently blocked because the MP's I've tried to lookat are unable to load their diffs from the librarian
<StevenK> jam: bzr di -r ? :-)
<jam> StevenK: the issue is you can't load a page like: https://code.qastaging.launchpad.net/~gz/bzr/2.4_robust_logging_714449/+merge/84035
<jam> because it thinks there was a diff generated for the MP
<jam> which should be sitting in the librarian
<jam> but the librarian id is for production
<jam> and doesn't exist on qastaging
<jam> so we're trying to force a new build
<jam> (by pushing to an old branch, and then running the scanner jobs)
<StevenK> jam: Right, so are you guys happy to beg gmb how to put up a deployment request?
<jam> StevenK: that would be good for us, yes
<StevenK> jam: I'm happy enough to do it as well -- but if gmb has it covered, that works for me.
<jam> jam: I'd like to expose our team to it, though we may need a bit more guidance if there are bits that gmb is unclear on
<gmb> StevenK, You assume I've ever done one; I haven't. Usually an Aussie has done the work before I come online :)
<StevenK> Haha
<StevenK> I'm happy to have a G+ hangout or something to walk all of you through it
<gmb> StevenK, That sounds like a good idea.
<StevenK> gmb: You're there in person, organise it and tell me where to be. :-)
<cjwatson> wgrant: It seemed kind of strange to go through series.getSourcePackage(spn).latest_published_component when I could just use source.component.  And there are different numbers of checks needed in each case, too.  I tried consolidating the two branches but it wasn't clear that it actually made things any more readable.
<cjwatson> wgrant: Is your concern purely style, or do you think I made a semantic mistake?
<jam> StevenK: we've managed to QA up to 15520, so I think we're ready to chat about deployment. I think gmb is setting up the hangout now.
<StevenK> jam: Okay, sounds excellent.
<StevenK> gmb: Please use my personal G+ account
<gmb> StevenK, We'll ping you a link in a sec... techofail happening atm.
<gmb> StevenK, https://plus.google.com/hangouts/_/1db0fa7779d1a1038578f306df3aa20ebf61a0be?authuser=0&hl=en-GB#
<StevenK> http://pastebin.ubuntu.com/1065642/
<StevenK> https://wiki.canonical.com/InformationInfrastructure/OSA/LaunchpadProductionStatus
<StevenK> https://devpad.canonical.com/~wgrant/production-revnos
* adeuring changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: adeuring | Firefighting: - | Critical bugs: 4.0*10^2
<jam> stub: It looks like the last dump of production => staging was back in April (04-20?). Is there a specific process if we want to get newer data?
<stub> jam: Yes, it was disabled and never reenabled after the disk space issue was worked around.
<stub> jam: I left it disabled, as I was going to try the new rebuild process on the weekend.
<stub> jam: I could do it now, but unfortunately the new rebuild needs to happen in place due to disk space limits, so staging will be offline for several hours while it works.
<jam> stub: so it is likely to be re-enabled Soon (tm) ? (I don't need it urgently at all, I was just trying to understand the status)
<stub> jam: Yes, I will run the script myself tomorrow and schedule it for weekly rebuilds on the weekend.
<jam> sounds good
<jam> stub: if we do fall back to the production librarian, can you help me debug things a bit?
<jam> If I try to go to an old merge proposal, such as: https://code.qastaging.launchpad.net/~gz/bzr/2.4_robust_logging_714449/+merge/84465
<jam> I get this oops: https://oops.canonical.com/oops/?oopsid=OOPS-106e5fe837de46e8a46356f148ecd888
<jam> which looks like a librarian lookup failure
<stub> jam: That is indeed a librarian lookup failure. So we need to work out why that isn't in the Librarian. I'll check the production db
<stub> (faster than tracking down the traceback from the librarian)
<jam> note, that wasn't the only merge proposal I tried, it happened on at least one other one.
<jam> stub: I can see the entry in 'libraryfilealias' in staging at least.
<stub> jam: It is there on production too
<stub> jam: You need to check qastaging's config to confirm that indeed requests are being passed back to the production librarian, and get the oops from the librarian for that request (grepping for that integer id should work)
<stub> This Librarian is oopsing now rather than spewing to a log file.
<stub> c/This/I think that the/
<jam> stub: would that be in launchpad-lazr.conf under 'librarian_server' ?
<jam> because all I see is "launch: True"
<stub> upstream_url or something
<jam> ah, there is another page to look at (qastaging-lazr.conf)
<jam> stub: upstream_host: launchpadlibrarian.net
<stub> So it is supposed to work
<stub> It might be restricted resources have a problem, qastaging librarian having troubles with the session database or something, but I would expect more fallout if that was the case.
<jam> stub: is there an obvious way to turn a lfa.id into a URL?
<jam> I tried poking at the code, but I also get a 404 when I go to the url I expected.
<stub> You need the lfa and filename
<jam> stub: which you get from 'SELECT id, filename FROM LibraryFileAlias where id = ....'
<stub> def get_libraryfilealias_download_path(aliasID, filename):
<stub>     """Download path for a given `LibraryFileAlias` id and filename."""
<stub>     return '/%d/%s' % (int(aliasID), url_path_quote(filename))
<jam> yeah, that is the one I tried
<jam> stub: I wonder if the value in qastaging differs from the one in production?
<stub> If it does, that would be a bug.
<stub> I get a 404 too.
<stub> So maybe the file never arrived on disk?
<jam> stub: well if you go to the production page, it looks correct: https://code.launchpad.net/~gz/bzr/2.4_robust_logging_714449/+merge/84465
<jam> but maybe the page has  a different diff now in prod
<stub> yer, mebbe
<stub> Is this one example of many, or is it a rare case?
<jam> stub: I tried 2-3 mp's and all of them failed in a similar way
<jam> stub: if I go to: https://code.qastaging.launchpad.net/bzr/+activereviews
<jam> all of the links off of there seem to fail with lookup error
<jam> stub: having tried 6 of them
<jam> stub: and all of them fail for https://code.qastaging.launchpad.net/launchpad/+activereviews as well
<jam> (one succeeds, but because it doesn't have a diff to show.)
<jam> I'm off to lunch for now.
<jam> but it is definitely systematically broken
<stub> jam: I'm thinking that the file exists on production, but you don't have access to it there so are getting a 404.
<stub> jam: I'm not sure if in this case your permissions on production or your permissions on qastaging are in effect.
<stub> jam: If a MP has been superseded, then perhaps the old one is no longer visible to anyone? So a superseded diff on production will always return 404?
<jam> https://oops.canonical.com/oops/?oopsid=OOPS-00746fbd7a0bd9d6b5273c8342091655
<jam> jelmer: ^^
<jelmer> gmb: lp:~jelmer/launchpad/skip-the-skips
<jelmer> gmb: lp:~jelmer/launchpad/skip-the-skips
* bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: adeuring,bac | Firefighting: - | Critical bugs: 4.0*10^2
<jcsackett> jelmer: saw your updates on the bzr update branch. r=me, and thanks for the answers.
<jelmer> jcsackett: thanks!
<czajkowski> jcsackett: sorry I didnt put you down for holidays, I go by whats on the admin in case folks are doing a swap day
<deryck> adeuring, https://plus.google.com/hangouts/_/355d337a416990bcc886b192858066f0198beb37?authuser=0&hl=en
<jcsackett> czajkowski: i forget there are holidays. :-P
<czajkowski> well indeeed but some people use them for swap days
<jcsackett> so no worries at all. :-)
<jcsackett> czajkowski: yeah--what your doing makes sense. those of us who forget to list the holiday can always reply to the staffing email with corrections.
<jcsackett> ...or our bosses can. :-P
<czajkowski> jcsackett: indeed
<jcsackett> sorry if my forgetfulness causes you any problems.
<czajkowski> jcsackett: no none at all
<czajkowski> just dont want you thinking I'm slacking or forgetting you either
<jelmer> mgz: pqm_email = Canonical PQM <launchpad@pqm.canonical.com>
<mgz> wgrant: I am a tool, my apologies
<jcsackett> czajkowski: the thought never crossed my mind. :-)
<wgrant> cjwatson: I believe it's a semantic mistake. You're using the source component, when the permission check should be on the target.
<wgrant> Shouldn't it?
<wgrant> jam: Restricted librarian files aren't proxied back from production, so you can't really view old MPs on staging
<cjwatson> wgrant: If that's so, then the current code is buggy too, because it gets the most recently published component for the series regardless of which archive it's in
<cjwatson> *for the source package in the distroseries
<wgrant> cjwatson: Yes, but a lot of code is crap, so it's not inconceivable.
<cjwatson> Well, yes, but it makes it hard to tell whether I'm making things any worse.
<wgrant> It would never really be noticed, since non-distro archives only publish in main.
<wgrant> mgz: What did you do?
<cjwatson> So what's the right fix?  Explicitly look up the latest published component in the target archive, but what if it isn't in the target archive at all yet?
<wgrant> There's rules for that already in archiveuploader.
<wgrant> strict_component exists for that reason
<wgrant> IIRC it tries with the source component and strict_component=True
<wgrant> If there is no source component, it tries with strict_component=False
<wgrant> To allow any component uploader to upload a new package
<wgrant> IIRC
<cjwatson> Dear nascentupload, thanks for not explicitly tagging when you pass a keyword argument so that I can't grep for it.
<cjwatson> But OK, that's similar to what half of check_copy_permissions is doing then.
<wgrant> Keyword-only arguments cannot come soon enough :)
<wgrant> I haven't actually read check_copy_permissions
<wgrant> I possibly should
<wgrant> Ah, right.
<gmb> jam, jelmer, vila, mgz, gary_poster: Behold: Python 2.7 parallel testing without an unknown worker (all the problem tests have been nuked for now; a proper fix is coming).
<wgrant> So the version before your branch is correct, apart from the fact that latest_published_component only considers distro archives, which won't be a problem in practice, but we should probably fix.
<gmb> http://ec2-184-72-186-58.compute-1.amazonaws.com:8010/builders/lucid_lp/builds/2
<vila> gmb: \o/
<gary_poster> sweet, gmb!  congrats and thanks!
<gmb> gary_poster, Welcome. Actually, much kudos must go to the Blue Squad and their genetic subunit parsing abilities.
<wgrant> 20 workers? Madness.
<gmb> Sparta.
<gary_poster> heh
<gary_poster> yay blue squad! :-)
<mgz> wgrant: read bug 692357 then filed bug 1018905 when we were fixing it anyway
<_mup_> Bug #692357: lib/canonical/lazr/doc/timeout.txt hangs on Python 2.7 <python-upgrade> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/692357 >
<_mup_> Bug #1018905: lp/services/doc/timeout.txt hangs in Python 2.7 <python-upgrade> <qa-untestable> <Launchpad itself:Fix Released by jameinel> < https://launchpad.net/bugs/1018905 >
<mgz> and lo, even named the bug (nearly) identically
<wgrant> mgz: Ah, yeah, I thought I'd filed that one.
<cjwatson> wgrant: OK, I'll see about fixing that today then.  Relatedly, do you think I might be able to try turning soyuz.derived_series.max_synchronous_syncs back down to something smallish and turning on soyuz.copypackageppa.enabled, both on dogfood, so that I can see how it behaves with current code?
<cjwatson> I think I know of two bugs right now: private->public copying isn't hooked up on the copy_asynchronous path, and I'm fairly sure that failed copy notification is about 95% implemented but doesn't entirely work.
<wgrant> cjwatson: Sure, that's fine on DF
<wgrant> cjwatson: It's never been tested to any significant extent, however.
<wgrant> So be prepared for spectacular fireworks.
<cjwatson> Any particular suggestions on exercising it?  I found the latter bug while experimenting with the test suite ...
<cjwatson> The actual copies themselves should be much the same as Archive.copyPackage, it's the UI glue that's probably broken
<cjwatson> AFAICS
<wgrant> Yeah, probably.
<cjwatson> I have a branch that deletes synchronous copies and that passes tests.  So I want to know what I'm missing :-)
<cjwatson> I note also that at the moment +copy-packages will copy from private to public PPAs using delayed copies without (AFAICT) any particular explicit confirmation, at least if I'm reading the code correctly.  Is this by design?
<cjwatson> (Because if it is then I can just add unembargo=True to fix the first bug I mentioned above.)
<jam> cjwatson: I see you are stealing all the low-hanging LoC for yourself :)
<cjwatson> I reckoned that if nobody had noticed in four years it couldn't have been *that* low-hanging
<cjwatson> wgrant: Did my work on https://code.launchpad.net/~cjwatson/launchpad/custom-uefi/+merge/111626 to use publisher configuration instead of lazr config look sensible to you?
<sinzui> rick_h_, deryck, jcsackett: I located the madness in choicesource tabbing. There was a handler setting the close button to have focus when the overlay had focus...the overlay is modal, it always has focus when visible
<sinzui> rick_h_, deryck, jcsackett: I have a plan so clever you could stuff it down your pants can call it a weasel. I wrote a handler to watch what was happening, then adapted it to make tabs cycle through the actions...you cannot tab out of the overlay. I can push this down to pretty overlay so that you cannot accidentally tab out of any overlay.
<deryck> sinzui, why not just remove the bit that calls focus to the close button?
<sinzui> I did
<sinzui> deryck, once to tab or shift tab out of a modal overlay, the focus is lost.
<sinzui> chromium changed the focus to the location bar
<deryck> sinzui, ok, rick_h_ and I are on a call together now.  we can chat more in a second about it.
<deryck> sinzui, ah, that's what I would expect.
<mgz> any ideas why I'm missing /var/tmp/mailman after doing normal launchpad install steps, have some failing test due to that which aren't 2.7 related
<jam> Have a good weekend, everyone!
<czajkowski> jam: toodles
<gmb> jam, jelmer, mgz, vila: The build failed... with only one failed test. http://ec2-184-72-186-58.compute-1.amazonaws.com:8010/builders/lucid_lp/builds/2/steps/shell_9/logs/summary
<gmb> But
<gmb> There were only ~2300 tests.
<gmb> Which means we're missing ~15,000 tests.
<gmb> Somewhere...
<sinzui> mgz, are these timeouts on doctests?
<sinzui> mgz we disabled the MailmanLayer because a few doctests are fundamentally flawed. They always timeout.
* adeuring changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: bac | Firefighting: - | Critical bugs: 4.0*10^2
<cjwatson> wgrant: https://code.launchpad.net/~cjwatson/launchpad/fix-check-copy-permissions/+merge/112832 - does this look better?
<jelmer> gmb: that bug was the one that vincent fixed earlier
<jelmer> gmb: s/that bug/that failure/
<czajkowski> evening
#launchpad-dev 2012-06-30
<cjwatson> wgrant: Urgh, test failure on fix-check-copy-permissions in lp.registry.browser.tests.test_distroseries.TestDistroSeriesLocalDifferences.test_sync_error_no_perm_component, which turns out to be because my change fails to do a proper ancestry check so if you copy into updates it doesn't realise that it has to check against release.  (Well, actually, the test creates a publication in backports; I haven't decided yet whether ...
<cjwatson> ... I think the test needs to be changed, but regardless the code is clearly still wrong.)
<cjwatson> wgrant: So I need to do an ancestry check.  There are at least two copies of getSourceAncestry already, one in NascentUpload and one in PackageUploadSource, but I don't have the right context to be able to use either of them.  I'm loath to add yet another copy of essentially the same logic.
<cjwatson> wgrant: Do you think it might make sense to move PackageUploadSource.getSourceAncestry somewhere else (where?  maybe under lp.soyuz.adapters?), since it doesn't actually need the PUS directly, just archive/DS/pocket/name?
<cjwatson> Actually, NascentUpload.getSourceAncestry is a closer match, looking at it, because it doesn't fall back to the primary archive.  Upload permissions to the primary archive shouldn't confer upload permissions to a PPA, and likewise for copying.  PUS.getSourceAncestry makes more sense for diff generation.
<wgrant> cjwatson: Can they sensibly be merged and just take a sequence of archives instead?
<cjwatson> Possibly.  PUS.gSA has a puzzling thing where it falls back to any DS in the primary archive but not in the context archive.
<cjwatson> And NU.gSA still uses the old DS.getPublishedSources, which doesn't make things any less confusing.
<cjwatson> Uh, and PUS.gSA takes the *last* of its sequence of lookups that it finds, not the first.
<cjwatson> Oh, no, it doesn't, it's just confusingly written.
<cjwatson>         for blah:
<cjwatson>             try:
<cjwatson>                 ancestry = ancestries[0]
<cjwatson>             except IndexError:
<cjwatson>                 continue
<cjwatson>             break
<cjwatson>         return ancestry
<cjwatson> Hi, dear PUS.gSA, how about "return ancestries[0]", love Colin
<cjwatson> So if you upload packages to a PPA for both precise and quantal, then the quantal PPA package will be diffed against whatever's in quantal in the primary archive and failing that whatever's newest in the primary archive, but never against the precise PPA package.
<wgrant> Yeah
<wgrant> That's a bit odd.
<cjwatson> I suppose that makes some kind of sense if the goal of PPA diffs is to work out what would happen if they landed in the primary archive.
<wgrant> Potentially deliberate
<cjwatson> I'm not sure I feel confident about merging these two, but renaming PUS.gSA to getSourceAncestryForDiffs might be a plan.
<wgrant> A good start, indeed
<cjwatson> Because honestly argh.
<cjwatson> Still not sure I know a good place for NU.gSA.  Is adapters a sane place for stuff that doesn't really have a single context object?
<wgrant> cjwatson: There, or maybe PublishingSet.
<cjwatson> Ah, I hadn't thought of *Set.
<cjwatson> That has a getNearestAncestor that is also not what I want.
<cjwatson> Wait.  SPPH.overrideFromAncestry will look for ancestry in any pocket.
<cjwatson> So, if I'm reading this right, if you only have universe component upload permissions, you can upload a package to backports that only exists in updates/main, but it will then be automatically overridden into backports/main.
<cjwatson> Maybe I need to go back to querying for publications in any pocket (which is what the previous code effectively did) just in order to get this permissions regression sorted out and unblock deployment, and leave an XXX comment to sort out the insanity separately.
<cjwatson> Because I'm really not entirely convinced by any of the three different implementations of this logic.
<wgrant> Indeed... :/
<cjwatson> wgrant: http://paste.ubuntu.com/1067513/ ?
<cjwatson> Oh and I need to fix an interface there too.
<cjwatson> So http://paste.ubuntu.com/1067517/
 * cjwatson should go and clean house instead of cleaning Launchpad ...
<Bert_2> can I ask questions here about issues I have with the launchpad code ?
<rick_h_> Bert_2: yea, most people are afk during the weekend, but feel free to ask
<Bert_2> yeah, I'll just ask ;)
<Bert_2> I'm trying to run my own instance of launchpad
<Bert_2> well, we, I'm working for a students developer group
<Bert_2> anyway, we have launchpad running using the rocketfuel script and the make schema make LISTEN... install make run thing
<Bert_2> but the information on the wiki only describes how to make it available to people who then have their dns changed so they can access it from launchpad.dev
<rick_h_> right, you need some DNS to tell everywhere where it's at.
<Bert_2> however, I want to use an actial domain for this instance of launchpad (something like launchpad.ulyssis.org)
<Bert_2> yeah, that doesn't work :S
<rick_h_> ok, so you'll need to adjust the apache conf file to be listening to that address
<Bert_2> launchpad won't show :S
<Bert_2> apache listens on it
<Bert_2> but doesn't display launchpad, but the it-works page :S
<rick_h_> for that name?
<rick_h_> http://paste.mitechie.com/show/717/
<Bert_2> yeah, it listens on the direct IP, I told it to, but it displays it works :S
<rick_h_> all of those launchpad.dev need to be changed
<Bert_2> or isn't an IP a good test ?
<rick_h_> no, because apache is looking for names
<Bert_2> yeah, I've read that
<rick_h_> it's a named virtualhost for all of the various bits, launchpad, bazaar.launchpad, etc
<Bert_2> I tried adding stuff, but that didn't work
<rick_h_> so unless the name in the url matches the apache config it won't serve the request
<Bert_2> zo I can actually just s/launchpad.dev/launchpad.ulyssis.org/ ?
<Bert_2> *so
<rick_h_> yea, that's what I'd start with
<rick_h_> I've not actually run it on a different hostname, so not sure what else might break, but at least that part needs to happen
<Bert_2> k, I was thinking about that but it seemed a little too simple because launchpad is so custom :S
<rick_h_> well, only in ServerName and ServerAlias
<Bert_2> I'll give it a try tomorrow and I'll probably be back later then
<Bert_2> yeah, I know ;)
<rick_h_> stuff like  <Directory /var/tmp/bazaar.launchpad.dev/mirrors/>
<Bert_2> yeah, I know that ;)
<rick_h_> might not like being changed
<Bert_2> don't touch other stuff :p
<rick_h_> ok cool
<Bert_2> probably not, the process will be custom there
<Bert_2> ulyssis runs a hosting service for student IT projects etc. over multiple servers with apache, ldap, nfs, etc., so we have quite some experience :p
<rick_h_> cool, yea give it a shot and see if that gets your father along.
<Bert_2> we just didn't want to fool around with launchpad too much ;)
<rick_h_> heh, understand :)
<Bert_2> yeah, I'll give it a go ;)
<Bert_2> let's hope we'll get it sorted out, this'll be a great service to the students
<Bert_2> also: is it normal that launchpad.dev by default contains some projects ?
<Bert_2> (or at least displays some on the homepage)
<rick_h_> yea, when you run make schema, if it's in dev mode, it'll load some sample data
<Bert_2> how do I take it out of dev mode then ?
<rick_h_> saves us a ton of time hacking on launchpad to have some default users, projects, etc
<Bert_2> (I saw an error on)
<Bert_2> *on make run
<rick_h_> yea, it's part of the configs directory I believe
<rick_h_> right, in dev mode you'll get more useful error messages/etc
<rick_h_> which can be used against you by hackers in production, to tell your server directory layout, etc
<rick_h_> so you're looking for the setting "devmode"
<rick_h_> which is in the configs/development/launchpad.conf
<rick_h_> so you'd want to setup a configs/production/launchpad.conf and get it started off of that once you've got things figured out
<Bert_2> yeah, I did some searching but I couldn't find any useful docs on help.launchpad.net :S
<rick_h_> https://dev.launchpad.net/WorkingWithProductionConfigs
<rick_h_> might help some
<Bert_2> fail
<Bert_2> how did I miss that link XD
<Bert_2> where was it ?N
<Bert_2> where was it ?
<rick_h_> I just searched the wiki for production and it came up
<rick_h_> I think you'll want dev.launchpad.net more than help.
<Bert_2> yeah, sorry, I mean dev.launchpad.net
<Bert_2> I went around looking on the pages concerning building and running launchpad
<rick_h_> anyway, off to pick up some dinner so good luck and it'll be a little quiet over the weekend so sorry if it takes some time to get answers, but there are people available to help.
<Bert_2> the production config thing is nog linked there
<rick_h_> yea, search ftw
<Bert_2> can I change that myself, cause that seems like something that's missing
<Bert_2> no problem man
<rick_h_> hmm, probably not. but you might suggest a good edit and we can help on that
<Bert_2> I'm used to longer delays on IRC ;)
<rick_h_> only launchpad devs can edit
<Bert_2> sure, I'll be in touch
<Bert_2> see ya ;)
#launchpad-dev 2012-07-01
<wgrant> rick_h_: One thing to point out in future is the restrictive licensing of the trademark and images.
<rick_h_> wgrant: to Bert? ah ok cool.
<wgrant> rick_h_: Yes, because if he's trying to run it on a launchpad.* domain then he probably doesn't know he can't use the Launchpad name :)
<rick_h_> right, didn't think of it sorry. Do we have a checklist for the "run your own LP instance" anywhere I should refer to?
<wgrant> No.
<wgrant> There's only one other known production instance.
<wgrant> We tend to discourage running your own, because it's raaaaather difficult and one can usually use launchpad.net instead.
<cjwatson> Does http://paste.ubuntu.com/1070301/ make sense to anyone (I don't think I've touched any code anywhere near that)?  I'm running it through ec2 again just in case ...
<wgrant> cjwatson: I haven't seen that one before, but it's not inconceivable that it's spurious.
#launchpad-dev 2013-06-24
<xnox> mgz: do you happen to know where does udd has it's database deployed? is it still on multiple sqlite .db files, or using a postgres instance somewhere?
<mgz> it's just local still, I belive
<mgz> when vila was looking at using juju, he was trying various things
<mgz> but I don't think we changed anything with the running instance
<xnox> mgz: i see. I'm planning to implement "shoot myself in the head, requeue option"
<xnox> mgz: e.g. consider tags from the branch as the right ones, if I told you so. For the cases where slangasek pushed "rich" history instead of reusing "package-import" historic tags.
 * xnox will migrate/snapshot over databases from jubany and see if i can poke them locally to implement that.
<cjwatson> StevenK: https://code.launchpad.net/~cjwatson/launchpad/das-removechroot-api/+merge/171125 + http://bazaar.launchpad.net/~ubuntu-archive/ubuntu-archive-tools/trunk/view/head:/manage-chroot FYI
<cjwatson> (And yes, that is seriously the least foul way I could find to do an upload progress bar ...)
<cjwatson> No problems with an 18-minute or so chroot upload to prod from behind my crummy ADSL.
#launchpad-dev 2013-06-25
<StevenK> cjwatson: I've approved the review, and the API script looks brilliant too.
<cjwatson> Thanks.  I reckon it ought to be possible to do setChroot(data=fileobj, sha1sum=foo), but life was a bit too short to get that done just now ...
<cjwatson> That would make the progress bar a lot less painful.
<StevenK> cjwatson: I'm also sure that one of us will take great joy in killing scripts/ftpmaster-tools/manage-chroot.py and related friends after your branch is deployed.
<cjwatson> I have a branch locally for that :)
<cjwatson> Though if you beat me to pushing, it's not like it's all that creative
<cjwatson> And you probably deserve the LoC credit more :)
<StevenK> wgrant: I think your Zope upgrading has broken auditorfixture bootstrapping: http://pastebin.ubuntu.com/5797569/
<wgrant> StevenK: Except I didn't touch auditorfixture
<wgrant> Was it not pinned?
<wgrant> That zc.recipe.egg constraint is suspiciously high
<wgrant> I'm pretty sure LP still uses 1.x
<wgrant> ztk-versions.cfg:zc.recipe.egg = 1.3.2
<StevenK> wgrant: Hm, it needs a versions.cfg?
<wgrant> StevenK: Might be advisable. It possibly picked up buildout 2 if I left that in download-cache
<wgrant> Or just specify an appropriate version constraint in setup.py
<wgrant> Pinning is probably less useful
<wgrant> You need to work out where that zc.recipe.egg > 2.0a3 constraint came from
<StevenK> wgrant: Reverting to the bootstrap.py in the tree: http://pastebin.ubuntu.com/5797584/
<wgrant> Right, you need to restrict your zc.buildout version
<wgrant>         $(SHHH) PYTHONPATH= $(PYTHON) bootstrap.py\
<wgrant>                 --setup-source=ez_setup.py \
<wgrant>                 --download-base=download-cache/dist --eggs=eggs \
<wgrant>                 --version=1.7.1
<wgrant> That's how LP invokes it
<StevenK> Error: There is a version conflict.
<StevenK> We already have: zc.buildout 1.7.1
<StevenK> but z3c.recipe.tag 0.7 requires 'zc.buildout>=2.0'.
<StevenK> Guess I need to clean harder
<StevenK> Hm, that doesn't help, but I can't see what is pulling in z3c.recipe.tag 0.7
<wgrant> StevenK: It will probably be trying to pick the latest version it can find
<StevenK> Oh, there is a 0.7 in download-cache
<StevenK> wgrant: Hmm, adding 'z3c.recipe.tag == 0.6', to setup.py in install_requires still has it grabbing 0.7
<wgrant> StevenK: I don't know how well that will work. It may need to be versions.cfg
<StevenK> % cat versions.cfg
<StevenK> [versions]
<StevenK> z3c.recipe.tag = 0.6
<StevenK> ...
<StevenK> Getting distribution for 'z3c.recipe.tag'.
<StevenK> Got z3c.recipe.tag 0.7.
 * StevenK stabs buildout
<wgrant> Is versions.cfg being used?
<wgrant> And are you sure there are no other version dependencies that are pulling in z3c.recipe.tag?
<StevenK> Stop being obstreperous, damn it!
<StevenK> wgrant: How do I check that?
<wgrant> -vv?
<StevenK> wgrant: Oh, it's direct, due to the buildout.cfg target [tags]
<wgrant> Sure, there's a direct dep
<wgrant> But your versions.cfg, if you've actually mentioned in buildout.cfg, should pin it to the right version
<wgrant> Unless some other dep wants 0.7
<StevenK> Oh, doh
<StevenK> cjwatson: I accidently manage-chroot.py. But removeChroot() is on prod.
<cjwatson> I noticed, thanks
<ttx> The Launchpad testr run takes a looong time. What would be the proper syntax to run only lib/lp/blueprints/tests/test_webservice.py ? I'm fumbling the -t PATTERN apparently
<cjwatson> bin/test -vvct lp.blueprints.tests.test_webservice
<ttx> cjwatson: are you on the Launchpad team those days or what ? :)
<ttx> cjohnston: thx
<ttx> oops
<ttx> cjwatson: thx
<cjwatson> not really, just an interested onlooker who sometimes finds it quicker to fix his own bugs :)
<cjwatson> though I suppose https://dev.launchpad.net/Contributions might belie that slightly
<ttx> Hmm... Looks like I could use some help. I'm working on exposing proposeGoal/acceptGoal/declineGoal methods in the specification API
<ttx> I think I got the implementation right but i'm strugging with unit tests
<ttx> See https://code.launchpad.net/~ttx/launchpad/lp1193389
<ttx> spec.proposeGoal(goal=series) in the tests fails with 401 Unauthorized
<ttx> ('goalstatus', <Specification 13 u'name-100429' for u'product-name-100423'>)
<ttx> It feels like it doesn't like that the proposer (or driver) plays with goalstatus through proposeGoal().
<cjwatson> I think you need to move those methods to a new ISpecificationDriver interface and declare that as requiring launchpad.Driver
<cjwatson> I think you need to move those methods to a new ISpecificationDriver interface and declare that as requiring launchpad.Driver
<xnox> udd import script getting 401 from launchpad. Are "natty" branches frozen or something?
<xnox> http://package-import.ubuntu.com/status/eglibc.html#2013-06-25 06:22:30.579024
<cjwatson> At the moment they only seem to be protected by browser code
<xnox> http://package-import.ubuntu.com/status/eglibc.html
<cjwatson> Also there's nothing in lib/lp/blueprints/configure.zcml that declares permissions on goalstatus, so it'll default to closed
<cjwatson> There's a comment about that though, but a strange one; goalstatus' security is going to be checked even if it's set via proposeGoal etc.
<cjwatson> You may need somebody more expert than me to look at this
<ttx> cjwatson: yeah, the model around series goal is a bit weird.
<cjwatson> Well, never mind model, the permissions seem mad
<cjwatson> Which suggests to me I don't fully understand what's going on
<ttx> there is also a comment in .zcml : "NB: goals and goalstatus are not to be set directly, it should only be set through the proposeGoal / acceptBy / declineBy methods"... so you'd think the method access control should be enough
<cjwatson> that was what I referred to above
<cjwatson> 15:55 <cjwatson> There's a comment about that though, but a strange one; goalstatus' security is going to be checked even if it's set via proposeGoal etc.
<cjwatson> simply going through a call to .proposeGoal is not enough to magically strip access control off zope attribute access
<ttx> cjwatson: ok
<cjwatson> but that means I don't understand exactly why it's failing for you right now, because my argument is that this is dangerously open at the moment :)
<cjwatson> I do think that the permissions on browser methods should be carefully unified with those on the underlying interfaces as a prerequisite to exporting things over the API.  Why the test is failing, not entirely sure
<cjwatson> Or, rather, I think I understand why it's failing but I then don't understand why the normal browser view code works
<cjwatson> It doesn't seem to be doing removeSecurityProxy or anything nasty like that
<ttx> Yeah. unfortunately we are quickly approaching the end of the timebox I can dedicate to fumbling my way through launchpad code... so let's see if someone can shine some light on this... otherwise i'll probably work around the missing feature from browser rather than API
<cjwatson> wgrant or StevenK should be able to answer when they show up
<cjwatson> Your change looks structurally roughly right to me, FWIW, it's just the legacy permissions weirdness ...
<ttx> cjwatson: heh, so close and yet so far :)
<wgrant> ttx, cjwatson: Managed to get a traceback out of that branch. It's lp.blueprints.subscribers.specification_goalstatus assuming that it will always get an unproxied object. I don't think that subscriber is ever actually used normally, since it's only triggered when distroseries or productseries on the spec change, and the only view that does that is SpecificationGoalProposeView, which doesn't call updateContextFromData, so doesn't fire an ...
<wgrant> ... ObjectModifiedEvent.
<wgrant> The model code seems to handle the goalstatus transitions correctly; I suspect you could just drop that subscriber entirely, since it's clearly broken
<wgrant> Some of that subscriber still shows sabdfl in annotate, so it hasn't been touched in a while :)
#launchpad-dev 2013-06-26
<StevenK> allow-picked-versions = false now complains about setuptools 0.7.4 no matter what versions.cfg states.
 * StevenK stabs buildout for being obtuse and obstreperous.
<StevenK> wgrant: I've been victorious over buildout
<wgrant> Great
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/export-pu-auditor/+merge/171460
<wgrant> StevenK: Do the bulk auditor load calls also bulk load the relevant people?
<wgrant> Are the auditor calls logged in the statement log?
<wgrant> What are the timeouts on the auditor calls?
<StevenK> wgrant: I don't think the enterpriseid stuff is that smart
<wgrant> It'll need to be
<StevenK> wgrant: Have you read enterpriseid_to_object() ?
<wgrant> StevenK: Not in maybe a year. Why?
<StevenK> wgrant: Because I can't think of a way given a list of enterpriseids to objectify them in one or two queries
<wgrant> StevenK: That doesn't obviate the need to do it in one or two queries.
<StevenK> Certainly not
<wgrant> You'd want to split them up by class, then bulk load
<wgrant> You'd also ideally introspect the storm cache to grab those that exist.
<wgrant> But that might be a bit difficult
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/sha256-archiveuploader-pre/+merge/171464 https://code.launchpad.net/~wgrant/launchpad/sha256-archiveuploader/+merge/171465
<StevenK> wgrant: r=me for the pre branch
<StevenK> wgrant: And slamming you with Needs Fixing on the second
<wgrant> That was more approve with comments material, but OK :)
<wgrant> StevenK: Updated
<StevenK> wgrant: r=me
<StevenK> wgrant: Are you going to bend gina to your will in terms of hashes, too?
<wgrant> No comment
<wgrant> But I might look at switching it to use some of the archiveuploader infrastructure
<wgrant> Rather than reimplementing the world
<StevenK> Sounds like a good idea
<ttx> wgrant: ok, I'll try to drop the subscriber and see how it goes :)
<ttx> wgrant: looks like it's passing tests now: https://code.launchpad.net/~ttx/launchpad/lp1193389/+merge/171491
<wgrant> ttx: Looking good, thanks. I'll review it properly tomorrow.
<ttx> wgrant: awesome, thx
#launchpad-dev 2013-06-27
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/bulk-eid-to-obj/+merge/171705 should address one of your concerns with export-pu-auditor
<wgrant> StevenK: Looks pretty good, but needs a few fixes
<wgrant> StevenK: Also, person merges
<StevenK> wgrant: Person merges?
<wgrant> StevenK: Person merges
<wgrant> .
<StevenK> wgrant: Repeating yourself does not help with more context/
<StevenK> s/\//./
<wgrant> StevenK: Person merges!
<wgrant> How do they interact with your branches?
<StevenK> wgrant: In terms of the eid branch, we can't do anything. If a person is converted to an eid and stored in auditor, and then merged into person, we will return the first person because we store by id
<wgrant> Why can't you do anything?
<wgrant> There's a well-defined successor to a merged person
<wgrant> person.merged
<StevenK> Ugh
<StevenK> Why can't we have simple things
<StevenK> This function already hurt my brain, and then I had to twist it to the side to understand your issues about its duplication, and now I have to deal with person merges.
<wgrant> What duplication?
<StevenK> "These are redundant. type_ids[foo] is just obj_id_to_eid[foo].keys()."
<wgrant> Oh, that thing
<wgrant> Do you understand now?
<StevenK> Yes, I've fixed it.
<wgrant> They are equivalent, except that type_ids[foo] has duplicates
<wgrant> Right
<StevenK> Tests even pass.
<StevenK> wgrant: http://pastebin.ubuntu.com/5803604/
<wgrant> Ah
<wgrant> Now that has a slight behaviour change
<wgrant> Which you probably want a test for
<wgrant> What happens when you ask for an ID that doesn't exist?
<wgrant> In the current version on LP it will give you None
<wgrant> In your new version it will omit the key
<wgrant> In another version it may crash
<StevenK> Yeah, that was probably why I had it at the top, but you complained.
<wgrant> Also, scheme is annoying
<wgrant> I'd say something like "instance, cls, id = eid.split(':')"
<wgrant> I think that's what they are
<wgrant> But I can't tell right now, because scheme is rather opaque :)
<wgrant> StevenK: I think omitting them from the response is probably best, so your new behaviour is correct
<wgrant> But we likely want a test for that
<wgrant> So we don't accidentally start crashing if an object disappears
<StevenK> But then auditor client will ask for them and then give KeyError
<wgrant> Isn't this on the other side of auditorclient?
<wgrant> We're parsing its responses
<wgrant> Not giving it things to ask for
<StevenK> Yes, we're parsing its responses
<StevenK> So, we get back lp:PackageUpload:6 as the eid for obj
<StevenK> Then the auditorclient calls enterpriseids_to_objects() and it omits lp:PackageUpload:6 because it doesn't exist. It then tries to map it to an object and crashes
<wgrant> Yes
<wgrant> Because enterpriseids_to_objects cannot know what behaviour the callsite wants
<wgrant> It has to be the callsite's responsibility to handle a missing object
<StevenK> Oh, I can use get anyway
<wgrant> Of course
<wgrant> That's probably the correct course of action for that particular case
<StevenK> wgrant: http://pastebin.ubuntu.com/5803623/
<StevenK> In terms of person merges, I have NFI
<wgrant> StevenK: existant isn't a word, and that assertEqual formatting is odd, but otherwise that looks good
<wgrant> Person merges don't belong in this branch
<StevenK> wgrant: That MP is updated.
<wgrant> StevenK: k
<StevenK> wgrant: http://pastebin.ubuntu.com/5803648/ should resolve another of your export-pu-auditor issues
<StevenK> Oh, I didn't drop scheme
<wgrant> I'd also prefer those two maps to be renamed as I said in my first comment, but it's at least a bit less confusing now
<StevenK> wgrant: scheme death, and maps renamed: http://pastebin.ubuntu.com/5803653/
<wgrant> Right
<StevenK> wgrant: One more with feeling for that MP.
<StevenK> GAH
<StevenK> YUI 3.10.3 was released 2 days after 3.10.2
<wgrant> Any notable fixes?
<wgrant> We don't care about tracking the x.x.1 increments unless they have relevant fixes
<StevenK> wgrant: The .swf vuln fixed in 3.10.1 snuck back into .2
<StevenK> So they fixed it again for .3
<wgrant> Hah
<StevenK> I don't think we serve any of the two swfs in our build tree
<StevenK> But convoy will probably return them if asked
<StevenK> 3.11 looks to have some nice speedups, too
 * StevenK waits for YUI 3.11 JavaScript for Workgroups
<StevenK> wgrant: Do you approve of the auditorclient diff?
<wgrant> StevenK: Sounds reasonable
<ttx> wgrant: applied your requested fixes, except the use of ISpecificationTarget, see my comment at https://code.launchpad.net/~ttx/launchpad/lp1193389/+merge/171491
#launchpad-dev 2013-06-28
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/auditorclient-timeline/+merge/171942
<wgrant> StevenK: Does that function correctly when given multiple objects, operations or actors?
<StevenK> wgrant: Sprinkling an assert in the multiple test: reference = None
<StevenK> actual    = "Object: ['lp-development:Person:243653', 'lp-development:Person:243654']; Operation: ('person-deleted', 'person-undeleted'), Actor: None"
<wgrant> Right
<StevenK> wgrant: Do you want that assert left in, or you wanted to see it cope?
<wgrant> Confused
<wgrant> Oh, by "wanted to see it cope" you meant "wanted to see it as a one off"?
<wgrant> I don't much care.
<StevenK> Yes, I did
<StevenK> wgrant: Could I get a review, then?
<StevenK> Or have you been defeated by connection limits?
<wgrant> s/connection limits/questions/
<wgrant> StevenK: We would ideally hook into auditorclient more directly, but this will do for now
<wgrant> I think
<wgrant> It might not actually work, though
<wgrant> You should test
<StevenK> Oh?
<wgrant> The receive is also wrong
<wgrant> The action should finish once auditorclient returns
<wgrant> Not once everything is dereferenced
<wgrant> Overlapping actions are generally a bad idea
<StevenK> wgrant: http://pastebin.ubuntu.com/5806468/
<wgrant> StevenK: Right
<StevenK> wgrant: Any other concerns?
<wgrant> I don't believe so.
<StevenK> wgrant: That MP has updated.
<wgrant> And is now approved.
<StevenK> wgrant: Thanks
<StevenK> wgrant: Do you want to review https://code.launchpad.net/~stevenk/launchpad/export-pu-auditor/+merge/171460 then?
<wgrant> StevenK: What happens if I reject, accept, reject, and then accept an upload?
<StevenK> wgrant: The last event's actor will be returned
<wgrant> StevenK: Are you sure?
<wgrant> There's a limit=1 on the single receive call
<wgrant> But that doesn't quite work for the bulk one
<StevenK> Let me write a test
<StevenK> wgrant: Right
<StevenK> wgrant: Calling reject, accept, reject with 3 different users results in the following: http://pastebin.ubuntu.com/5806804/
<wgrant> StevenK: That shows that it happens to work at least sometimes
<wgrant> I want to know whether it *should* work, or whether that's an accident.
<StevenK> wgrant: Adding sleeps between the reject, accept and reject also returns the same returns.
<StevenK> Which shows that it isn't just working due to the 2 events having the same timestamp
<StevenK> s/same returns/same results/
<wgrant> StevenK: That's not very strong evidence :)
<wgrant> I'd really like to see eg. the request that goes to auditor, and the SQL behind it
<wgrant> postgres will often return things in insert order initially on small tables
<wgrant> Simply because of the likely physical layout
<wgrant> I don't see how this works reliably, so I am inclined to believe it does not.
<StevenK> wgrant: It's auditorfixture, so it's backed on sqlite
<wgrant> s/postgres/sqlite/
#launchpad-dev 2013-06-29
<sladen> https://bugs.launchpad.net/ubuntu/+source/octopussy/+bug/1055766  can somebody nuke the last five comments and kb the user
<_mup_> Bug #1055766: grep -R doesn't automatically search amazon <apport-bug> <i386> <oneiric> <octopussy (Ubuntu):Fix Released by ilovepussy> <https://launchpad.net/bugs/1055766>
<sladen> and ideally check if they hit anything else too
#launchpad-dev 2014-06-24
<xnox> https://code.launchpad.net/~xnox/lazr.authentication/fix-wsgi-intercept-0.6/+merge/224287
<xnox> https://code.launchpad.net/~xnox/lazr.restfulclient/py3/+merge/222172
<xnox> are all of above managed by tumble thing? ( tarmac?)
<wgrant> xnox: The trunks of both lazr.authentication and lazr.restfulclient are manually managed, not by Tarmac or PQM.
<wgrant> xnox: What's with the weird spacing changes in your wsgi_intercept branch for lazr.restfulclient?
<xnox> wgrant: hehe. wsgi_intercept 0.5.x -> 0.6.x series wraps tracebacks into it's own exception thus interractive raised expection textual representation is now different
<xnox> wgrant: it does look odd, should i be fixing wsgi_intercept to print stuff as it used to?
<xnox> the 0.6.0 came out in Novermber 2013.
<wgrant> xnox: I'd quite like to know why those spacing changes happen; they look a lot like a bug.
<xnox> wgrant: yeah, running without any ...., all headers are printed without space after colon
<wgrant>                 self.output.write(k + b':' + v + b"\n")
<wgrant> That sure looks buggy to me.
<xnox> yeah, that's wrong.
<wgrant> Trivial fix, too, fortunately :)
<xnox> i'll upload that into debian & propose upstream....
<xnox> can buildout use system-wide packages somehow ?
<wgrant> xnox: It *can*, but we prefer not to at this point.
<wgrant> We currently support running Launchpad on lucid, precise and trusty.
<wgrant> We backport some packaged things like python-debian, but it's often more trouble than it's worth.
<wgrant> (Particularly when it's a non-backward compatible change like this; we'd have to upgrade the package and Launchpad in lockstep)
<wgrant> Hm
<wgrant> So that header format isn't actually illegal. Whitespace isn't mandatory.
<wgrant> But I've never seen any non-terrible app not prefix the value with a single space.
<xnox> yeah, all dumps i've ever seen have a space.
 * xnox ponders if it is in-fact in the HTTP RFC or some such.
<wgrant> " The field value MAY be preceded by any amount of LWS, though a single SP is preferred."
<cjwatson> beat me to it
<cjwatson> was literally just selecting that text :)
<wgrant> Heh
<xnox> opened a merge proposal upstream, now making upload into debian.
<wgrant> Excellent.
<wgrant> That'll save a lot of trouble with fixing tests in lp:launchpad too.
<wgrant> "fixing"
<xnox> and all of openstack
<xnox> although they might not be doing doctests as much as launchpad does.
<xnox> wgrant: barry did review this one https://code.launchpad.net/~xnox/lazr.restfulclient/py3/+merge/222172 can you please merge it?
<xnox> or should barry do the merging?
<wgrant> xnox: Doing.
<wgrant> If GitHub's JS stops hanging my Firefox...
<xnox> wgrant: you merge lazr on github and then mirror to launchpad? *giggle*
<wgrant> Heh, no, was looking at your PR
<wgrant> xnox: I note a potential problem. The new wsgi_intercept doesn't seem to have the zope.testbrowser or mechanize support that the old one does, and Launchpad uses that.
<wgrant> So it'll work for lazr.restfulclient, but not Launchpad itself.
<xnox> wgrant: horum. I'm working on python3-launchpadlib at the moment, so that should be fine. ev kind of have vetoed for me to go into python3 launchpad project =)))))))
<wgrant> Heh
<wgrant> It's just a bit weird that we have to keep Launchpad using an old version, but I guess we'll live.
<wgrant> xnox: Can you set a commit message on https://code.launchpad.net/~xnox/lazr.restfulclient/py3/+merge/222172?
<xnox> meh, as long as there are no security issues....
<xnox> wgrant: done.
<wgrant> I don't trust that library enough to let it near production :P
<wgrant> It's test-only.
<ev> xnox: hey now, I said you can spend your time however you likeÂ â I'm not your manager â but there were seemingly better places to invest that time in Launchpadland.
<xnox> ev: yeah, rumour has it you are after python3-launchpadlib though ;-)
<wgrant> Definitely :)
<wgrant> Porting launchpadlib is pretty tractable once the lazr.restful test dep is pushed out-of-process.
<wgrant> lp:launchpad is just about impossible without splitting it up.
<xnox> ev: for some sort of steam-punk engine
<ev> :)
<wgrant> xnox: Landed, thanks.
<wgrant> Poke me if you need anything else.
<xnox> wgrant: my wsgi-intrcept fix was pulled upstream already =)
<wgrant> Nice.
<wgrant> cjwatson: That launchpad-buildd ltsp change seems a bit weird. Is ltsp in the closure that ubuntu-rtm will have, for example?
<cjwatson> We could make it i386-only in line with the old make-chroot.sh, I guess
<cjwatson> Though I suppose we'll still have i386 in rtm
<cjwatson> It is weird, I just couldn't think of anything better, suggestions welcome
<cjwatson> wgrant: Is the CSV file that appears to be written by lp:~wgrant/lp-ftbfs-report/production available anywhere for http://qa.ubuntuwire.com/ftbfs/ ?  I can't make the obvious URLs work, and I assume the main page is a server-side redirect of some kind
<wgrant> cjwatson: It's at the totally logical and obvious URL of http://people.ubuntuwire.org/~wgrant/rebuild-ftbfs-test/primary-utopic.csv
<cjwatson> Aha, thanks.
<cjwatson> I was wondering about exposing data over time somehow
<xnox> wgrant: may i request review/merge of https://code.launchpad.net/~xnox/lazr.authentication/fix-wsgi-intercept-0.6/+merge/224287 ?! that one seems wsgi_intercept genuinely staying spec compliant and lazr.authentication was not.
<xnox> please =)
<cjwatson> wgrant: Do you have any idea what the occasional build upload failures we see where the upload log just says "Server said:" are about?  I understand it's some kind of librarian glitch but not a lot more.  I've seen two so far with livefs builds, which is a fairly annoying rate
<cjwatson> (particularly without retry)
<wgrant> cjwatson: Hm, I guess it might be more common with livefses because they're bigger.
<wgrant> It's some librarian glitch which I've not been able to reproduce locally.
<wgrant> We should probably just fix it by making the client retry.
#launchpad-dev 2014-06-25
<cjwatson> Where does the librarian upload log go?  I can only find download logs on carob ...
<wgrant> cjwatson: Is there logging?
<wgrant> Except on error.
<cjwatson> wgrant: I was indeed looking for error logging on the off-chance
<wgrant> cjwatson: I think that should end up in librarian.log too.
<cjwatson> Hm, OK.  I was specifically looking for things around the time of a livefs upload failure, and found nothing.
<cjwatson> Which suggests that possibly the client just failed to connect.
<cjwatson> Working on making it retry a few times, anyway.
<wgrant> cjwatson: Did you consider just retrying the librarian communication, rather than recreating all the DB objects as well?
<wgrant> I guess all it will do is consume an extra LFA and LFC ID
<cjwatson> Oh, I hadn't noticed I'd done that
<cjwatson> I don't mind refactoring, drop me a reminder comment
<wgrant> Though, hm.
<wgrant> How does this work, I wonder?
<wgrant> The file is never seeked back to the start.
<wgrant> Oh
<cjwatson> *cough* apparently my tests are not as complete as you might hope
<wgrant> I guess it only handles failure on the _sendLine('')?
<cjwatson> it was meant to be throughout, mistake
<wgrant> Which is possibly all we can reasonably do; I'm not sure whether everything we read is seekable.
<cjwatson> the failures we're seeing from livefs uploads are from the response != '200' case
<cjwatson> so perhaps we can try to seek and if we can't then, well, we can't retry
<wgrant> Yeah, that works.
<wgrant> Oh, the _sendLine('') is explicitly trying to *not* read the error.
<wgrant> We just send the entire file blindly, as long as the server accepted the connection..
<wgrant> So yeah, that sounds sane.
<wgrant> Try to seek, fail otherwise.
<wgrant> I shall comment.
<cjwatson> FWIW I just had it on the same DVD livefs twice in a row
<wgrant> Ew.
<cjwatson> So maybe that's a good way to reproduce it
<cjwatson> https://launchpad.net/~ubuntu-cdimage/+livefs/ubuntu/trusty/ubuntustudio/+build/235
<cjwatson> It managed its amd64 sibling
<cjwatson> Maybe just a socket timeout or something?
<wgrant> Yeah, tempting to try that on DF at some point.
<cjwatson> Though I can't see what would be setting the timeout.
<cjwatson> wgrant: Hm, a thought.  What with all the moving around of librarian directories, what's to say that the librarian's "incoming" directory is on the same filesystem as the eventual target location?  We might well be incurring the slow path of shutil.move on every upload, which happens before the server replies to the client.
<cjwatson> We could defer opening LibraryFileUpload.tmpfile until as late as possible, at which point we'll generally know the content-id and be able to write the file to the same directory as its target location and thus guarantee that shutil.move is cheap.
<cjwatson> I suspect there might then be no need to complicate the client, but we'd be able to find out fairly quickly given the error rate on production.
<cjwatson> This is guesswork, but I don't see anything else in the store path that might be slow.
<wgrant> cjwatson: Have you been able to demonstrate that a timeout causes the behaviour we see?
#launchpad-dev 2014-06-26
<wgrant>   File "/home/wgrant/src/launchpad/branches/amd64/lib/lp/services/librarian/client.py", line 204, in addFile
<wgrant>     raise UploadFailed('Server said: ' + response)
<wgrant> lp.services.librarian.interfaces.client.UploadFailed: Server said:
<wgrant> I sent the librarian a whole lot of SIGUSR1 and SIGUSR2 during that
<wgrant> It was the first of my 50 or so tests to fail, so possibly not a coincidence.
<wgrant> Oh
<wgrant> That's just because the librarian silently dies on those signals rather than actually doing anything useful.
<wgrant> How do we rotate logs on prod...
<wgrant> Ah, it only hooks USR1 when there's actually a log specified, and USR2 only works on appservers.
<wgrant> ...
<wgrant> 2014-06-26 11:24:11+1000 [-] Received SIGINT, shutting down.
<wgrant> 2014-06-26 11:24:11+1000 [-] Main loop terminated.
<wgrant> 2014-06-26 11:24:11+1000 [-] Server Shut Down.
<wgrant> Segmentation fault (core dumped)
<wgrant> cprov: Good catch on the missing arg. I'd only dropped those extra log entries during testing.
<wgrant> Thanks for all the reviews.
<wgrant> s/missing/extra/
<dpm> wgrant, I seem to remember that enabling armhf builds for a PPA should be requested first. Would it be possible to enable them for https://launchpad.net/~dpm/+archive/ppa? Thanks
<wgrant> cprov: Oh, now I remember. I was going to delete handleFailure in the branch I'm working on now.
<cprov> wgrant: it works for me :)
<dpm> wgrant, ah, please ignore my previous message. It seems that ppa has already got armhf builds enabled
<wgrant> dpm: Yep, armhf already enabled there.
<dpm> yeah, sorry for the noise
<wgrant> np
<wgrant> yay, made it over 100 lines of credit again
<wgrant> cprov: https://code.launchpad.net/~wgrant/launchpad/handleStatus-crash/+merge/224617
<cprov> wgrant: on it
<cprov> wgrant: r=me
<wgrant> cprov: Thanks.
<wgrant> And it'll get a nice workout overnight on staging...
<cjwatson> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=751207 - perhaps we should pass DH_VERBOSE=1 to our builds?
<wgrant> Hmm
<wgrant> Indeed
#launchpad-dev 2014-06-27
<cjwatson> wgrant: Do you see anything horribly wrong with https://dev.launchpad.net/Soyuz/LiveFilesystems before I send it to plars by way of documentation?
<wgrant> cjwatson: Oh, I was just telling him a few hours ago that there was no documentation yet so he'd have to ask us when he ran into issues.
<wgrant> That's easier.
<cjwatson> I was going to reply to his mail with docs and then realised it would make rather more sense to put them somewhere.
<wgrant> s/livefsbuild/livefs_build/
<cjwatson> e ryes
<cjwatson> the link was right :)
<wgrant> Yep
<wgrant> Might want to mention that they execute with the archive's dependencies, too. Apart from that I think it's right.
<cjwatson> tweaked
<wgrant> We seem to have arm64 capability.
<wgrant> echan, but you get the idea
<cjwatson> yay
<wgrant> Not quick, but it does seem to work.
<cjwatson> Are we expecting qtcreator to work?  Can't remember if that uses threads in its build
<wgrant> I thought it did, but I might be wrong.
<wgrant> I was hoping it did.
<wgrant> stub: https://code.launchpad.net/~wgrant/launchpad/oauth-NOnce/+merge/224818 <- as we discussed a couple of weeks ago. let me know if you can think of any reason I'm wrong...
#launchpad-dev 2015-06-22
<lifeless> wgrant: what blew up?
<wgrant> lifeless: MySQL got very attached to keystone's token table and wouldn't let anyone else have a look.
<lifeless> eep
<lifeless> there's a incremental cleaner spamaps wrote
<lifeless> its pretty much the only safe way to clear stuff in that table
<wgrant> lifeless: This was a more restricted version of the query from "keystone-manage token_flush"
<wgrant> (though that query is itself pretty silly)
<lifeless> oh no
<lifeless> thats not the safe way
<wgrant> It's clearly not safe, but it shouldn't failed in quite the way it did.
<lifeless> wgrant: https://github.com/openstack/tripleo-image-elements/blob/master/elements/keystone/cleanup-keystone-tokens.sh
<wgrant> lifeless: Ah yes, why didn't I think to look there :)
<lifeless> wgrant: I know, I know
<lifeless> there's a story, of course
<lifeless> wgrant: well, I'd need to see the query you used (so I can show it to spamaps :P)) to see if what happened is expected.. but I've seen that pathology before
<wgrant> hahahahah
<wgrant> "Also, because the keystone tokens are somewhat random 64 byte strings, there will be gaps in the InnoDB key storage. In order to preserve transactional integrity while deleting these rows, InnoDB has to lock these gaps. So new tokens that fit into these gaps will have to wait for the entire delete to finish to be inserted."
<wgrant> srsly?
<lifeless> yes
<lifeless> its how mysql handles new keys in gaps in the index
<lifeless> to deal with range queries being invalidated
<lifeless> I forget the details on postgresql's handling of that (which IIRC happened in 9.0?)
<wgrant> Right, the new SERIALIZABLE transactions do range locking.
<wgrant> But not like that...
<wgrant> (unless something was actually doing range queries on the token value, which seems unlikely)
<blr> wgrant: will need to have a review/landing for product-setbranch first, but would you mind a quick sanity check on https://code.launchpad.net/~blr/launchpad/project-meta-go-import/+merge/262550 please?
<wgrant> blr: Should use vhost.mainsite.hostname rather than launchpad.non_restricted_hostname. secure_codebrowse_root also isn't usable here (you can't branch from it), so you want branch.bzr_identity to get an lp: URL instead.
<blr> wgrant: I'm not sure the lp: prefix will work in this context will it?
<wgrant> We also can't expose a git repository for productseries.
<wgrant> blr: It depends how go get parses it.
<wgrant> if it gives it to bzr or bzrlib, it will handle lp: fine.
<wgrant> But it may be excessively pedantic.
<blr> mwhudson: any idea there?
<blr> wgrant: err true, it only needs the bzr case for product series..
<wgrant> blr: If the productseries case isn't already supported by the manual launchpad.net bits of go get, we shouldn't support it either.
<mwhudson> there is supoprt in the go tool for lp:product/series
<mwhudson> (unfortunately)
<mwhudson> i don't entirely understand what it is doing :-)
<blr> mwhudson: would you be able to do a bit of digging to see if the remote import path will support lp:product/series?
<mwhudson> blr: what do you mean, exactly?
<mwhudson> i'm happy to dig, sure
<mwhudson> although, arg, regex
<blr> can I render `lp:product/series' rather than the http uri
<mwhudson> 		re:     `^(?P<root>launchpad\.net/((?P<project>[A-Za-z0-9_.\-]+)(?P<series>/[A-Za-z0-9_.\-]+)?|~[A-Za-z0-9_.\-]+/(\+junk|[A-Za-z0-9_.\-]+)/[A-Za-z0-9_.\-]+))(/[A-Za-z0-9_.\-]+)*$`,
<mwhudson> oh i see
<blr> e.g. <meta name="go-import" content="launchpad.net/product/series bzr lp:product/series">
<mwhudson> magic 8 ball says ask again later
<blr> mwhudson: fair enough
<mwhudson> seriously, not quite sure, will need to stare a bit harder at this
<mwhudson> i think it will need to be lp:// to avoid scheme pedantry :)
<mwhudson> otherwise ... i think it will probably work
<mwhudson> oh hrm, maybe not?
<mwhudson> blr: if you just make it say https://launchpad.net/product/series that matches what the go tool will do today
<mwhudson> and https://launchpad.net/product on the top level page
<mwhudson> i think
<blr> mwhudson: ok thanks
<blr> wgrant: in that case, is secure_codebrowse_root appropriate?
<blr> or is there an https equivalent of bzr_identity
<wgrant> hum, it relies on the webapp redirect behaviour? we should probably not.
<mwhudson> currently it does
<mwhudson> agree that's a bit daft and it would be nice to not do that i guess
<cjwatson> wgrant: Thanks for the testfix, BTW
<cjwatson> wgrant: Did you get a chance to look over https://code.launchpad.net/~cjwatson/launchpad/git-repository-ui-edit-target/+merge/261232 again, or should I just go ahead and land that?
<wgrant> cjwatson: Ah, sorry, wasn't here today. Will have a quick look.
 * cjwatson wonders what's up with https://code.launchpad.net/~wgrant/launchpad/git-import-stable and friends.  12-hour scans, it seems to find revisions but they aren't visible on bazaar.launchpad.net, even via sftp
<wgrant> cjwatson: They are not scans.
<wgrant> Their respective imports are incomplete, but continue 30,000 revisions at a time.
<wgrant> That is the cause of the grey rather than the green tick.
<wgrant> cjwatson: No big issues with the branch. Feel free to land it.
<cjwatson> imports> oh, right
<cjwatson> Thanks.
<wgrant> The scans should succeed after a retry or two, once the import is complete.
<wgrant> I manually did an incremental scan to get the initial one in.
<cjwatson> Explains why I couldn't find anything in ackee's bzrsyncd logs, anyway.
<cjwatson> Incremental scan?
<wgrant> Inserting the 130,000 Revisions, RevisionParents, etc. on top of the 130,000 BranchRevisions is simply too much.
<wgrant> So for initial scans of new big branches we have to do it gradually: push them a couple of thousand revisions at a time, so the Revisions aren't all created in one hit which times out.
<wgrant> Similar to the way imports work, except in smaller batches and manually.
<blr> morning
<blr> wgrant: working on GitRepositoryVocabularyOnProduct, have mostly copied the existing bzr class, but making some sense of it I think.
#launchpad-dev 2015-06-23
<blr> wgrant: how are the vocabulary strings resolved?
<blr> e.g. ProductSeriesView.branch exports a RefrenceChoice with attrib vocabulary='BranchRestrictedOnProduct' ..
<wgrant> blr: An IVocabularyFactory with the same name.
<wgrant> You'll see securedutility ZCML directives for that registration.
<blr> ah, yes I should probably remember to consult zcml config whenever something appears magica
<blr> magical
<wgrant> Yup
<wgrant> Magic usually means adapters, and adapters are registered in ZCML.
<cjwatson> blr: I picked up https://code.launchpad.net/~blr/launchpad/bug-1334577-verbose-diff/+merge/243751 and posted an updated review.  Should be possible to finish this up without too much more work, hopefully by end of month.
<cjwatson> wgrant: Any chance you could find a bit of time to look over https://code.launchpad.net/~cjwatson/launchpad-buildd/avoid-pbuilder/+merge/261989 ?
<xnox> cjwatson: wgrant: who is ~foli and why does he mess with ~ubuntu-dev members =)
<xnox> cjwatson: wgrant: would you be able to PM me his email address (none are published on the launchpad page) to raise an issue with his actions w.r.t. ACLs of ~not-canonical per package uploader =)
<cjwatson> xnox: He's a Canonical sysadmin
<cjwatson> xnox: More detail, please?
<xnox> cjwatson: so james hunt was deactivated from ~ubuntu-dev by foli, yet james hunt is a per-package uploader for e.g. upstart approved by DMB. And as far as I know DMB controls ~ubuntu-dev.
<xnox> seems like too much clean-up was done during exit procedure?
<cjwatson> I suspect that's an overenthusiastic script somewhere, indeed
<xnox> hence i wanted to email DMB about it, and CC him, to possibly give more insight or correct things.
<cjwatson> PMed
<xnox> thanks
<cjwatson> I'm reasonably sure you can just add back and it won't be done again, mind you
<cjwatson> But I just found the whitelist, and it contains ubuntu-core-dev but not ubuntu-dev, so that's almost certainly an error
<xnox> cjwatson: sent an email to DMB, with cc of james and the two people you have PMed me. Board also has enough canonical people to check that everything is in order.
<cjwatson> xnox: Might want to reference lp:canonical-is-scripts blessed/leavers/leaver.csv (which you won't be able to see directly, but I'm pretty sure that's the whitelist)
<xnox> cjwatson: meh, i'd rather be explicit such that things can be checked and corrected. I thing on grand scheme of things, this is trivially minimal thing.
<xnox> haha, ok.
<cjwatson> xnox: I'm pretty sure this is just because PPU is relatively rare so IS didn't previously notice that ubuntu-dev needed to be whitelisted directly
<xnox> true.
<cjwatson> hmm
<cjwatson> I think I might have accidentally put together a halfway reasonable side-by-side diff implementation in a couple of hours of effort
<cjwatson> hloeung: Do you need help finishing up https://code.launchpad.net/~hloeung/launchpad/openpgp-show-fingerprint/+merge/244614 ?
<blr> wgrant: morning
<blr> wgrant: added new browser tests (using BrowserTestCase rather than doctests) for product views - we didn't appear to have any.
<hloeung> cjwatson: yeah I do actually. I'm not even sure where to begin fixing those xx*-pdf.txt as mentioned
<hloeung> cjwatson: or ensure that my change doesn't break them
<wgrant> blr: Morning.
<wgrant> blr: BrowserTestCase is good for testing general cases, but lots of edge cases are nowadays better checked using create_initialized_view.
<cjwatson> hloeung: Do you have a working test environment?
<wgrant> blr: How close is it all to finished?
<hloeung> cjwatson: nope, I haven't done any LP development before
<cjwatson> hloeung: Would you like to get this sorted out so that you're better set up for the future, or do you just want to throw it over the wall to us and let one of us finish it off?
<blr> wgrant: have an issue with my RestrictedGitRepositoryVocabularyOnProduct test, trying to resolve now (everything else is finished)
<blr> cjwatson: thanks very much for looking at the verbose diff branch, will be good to get that squared away
<hloeung> cjwatson: I think it might be best if I throw it over the wall. That is if one of you guys are willing to take over it
<cjwatson> hloeung: sure, I can finish it up.  It's basically a matter of grepping for pks/lookup and applying the obvious URL changes
<cjwatson> hloeung: test_web doesn't fail, but the other mentioned ones do
<cjwatson> (But test_web should be adjusted anyway to make sure that testkeyserver can cope with the new form)
<cjwatson> Anyway, I'll submit a fixed/updated version tomorrow
<hloeung> cjwatson: thanks
<wgrant> cjwatson: Oh, fmt:ssdiff, nice.
<cjwatson> wgrant: http://people.canonical.com/~cjwatson/tmp/ssdiff-1.png
<wgrant> cjwatson: Did you consider splitting the code into two halves to make it more comprehensible? Specifically, have one function which yields categorised lines with diff and file line numbers, and then another which can just process them without all the weird continue stuff.
<blr> wgrant: cjwatson: point of confusion regarding valid repositories for a product, in the bzr case the vocabulary validates product.series.branch, however the git case sets the default repository - what property should have the vocab?
<wgrant> blr: The form field must have the vocab directly.
<cjwatson> wgrant: That might not be a bad idea.
<cjwatson> generators++
<wgrant> cjwatson: fmt:diff can probably be simplified based on that too.
<wgrant> Though it wouldn't actually use the in-file line numbers.
<cjwatson> They're structurally pretty similar, yeah.
<blr> wgrant: ah, so I _don't_ want a copy_field, but should just set the vocab on the TextLine
<wgrant> blr: Indeed, there's no corresponding model field for git, because it's controlled by IGitRepositorySet.setDefaultRepository
<blr> right
<blr> that makes sense, thanks.
<wgrant> blr: It wouldn't be a TextLine, but rather a Choice or something, I think.
<wgrant> blr: It should be just like ProductSeries.branch, except directly on the view.
<wgrant> cjwatson: I think the tests become a good bit less ugly when they can test edge cases without parsing HTML...
<wgrant> Though some of them do need to.
<cjwatson> Indeed.
<cjwatson> Could be fleshed out a bit without dying of boredom.
<wgrant> I'd probably even split out a third layer which yields (type, diff_lineno, added_lineno, removed_lineno, added_text, removed_text) without doing HTML, but that's possibly little benefit once the complex parser is split out.
<cjwatson> I'm going to claim that this was all part of the inline comments improvements requested by stakeholders, and not an epic exercise in procrastination, honest.
<wgrant> Heh
<wgrant> There is either a bug or a stakeholder request about side-by-side diffs.
<cjwatson> Both.
<cjwatson> I assigned both of them to me once I realised it was tractable.
<cjwatson> (loggerhead does the transformation in JavaScript, I think, which is sort of arguably a better UX since you can switch back and forward without reloading; OTOH at least loggerhead's implementation is dreadfully slow and the query string isn't in fact too bad.)
<cjwatson> (Also we could still flip it with AJAX later if we wanted.)
<cjwatson> blr: I also reviewed https://code.launchpad.net/~blr/launchpad/bug-114705/+merge/243320, just in case that's stuck way back in your inbox.
<blr> cjwatson: almost felt like leaving that open given who reported it :(
<cjwatson> yes ...
#launchpad-dev 2015-06-24
<wgrant> Oh, russkaya, go away.
<wgrant> Nobody likes you.
<blr> wgrant: cjwatson: what's the significance (if any) or 'scotty' and 'mountain'? heh
<blr> s/or/of/
<cjwatson> blr: Your guess is as good as mine; the branch vocabulary tests go back quite a way
<cjwatson> I probably copied them, but ...
<blr> cjwatson: I've perpetuated the mystery.
<blr> wgrant: all sorted save a GitDefaultConflict on setting the default repo to the same repo on a second submission.
<blr> wgrant: suppose that will have to handled with the abort hack?
<cjwatson> blr: That's a bug in GitRepository.set{Owner,Target}Default, I think; they should return early if existing == self
<cjwatson> blr: Please fix that while you're there rather than inserting abort hacks.  The test should be simple
<blr> cjwatson: ah, I assumed that was the right behaviour. Sure, I can fix that up.
<cjwatson> (Or just "if existing is not None and existing != self", given that the following code is quick and a no-op in both cases)
<blr> grabbing some lunch
<cjwatson> hloeung: All right, so much for tomorrow morning.  https://code.launchpad.net/~cjwatson/launchpad/openpgp-show-fingerprint/+merge/262803
<cjwatson> We'll get +activereviews under control yet :-)
<hloeung> heh; oh so stories/*/xx-*.txt aren't actually generated?
<cjwatson> No
<hloeung> it looks easy now that I've seen someone else's diff of the changes required
<cjwatson> Doctests are no fun
<cjwatson> But they're easy enough to do lightweight tweaks to if you have an environment to run the tests in, indeed
<cjwatson> The idea is that the indented bits that look like Python interpreter sessions are run and the output compared with what the file says it's supposed to be
<cjwatson> So we wouldn't want to generate them
<wgrant> Doctests are good for some things :)
<wgrant> Mainly for documenting APIs.
<wgrant> View aren't APIs :/
<lifeless> doctests are the devils spawn
<lifeless> there's a very narrow set of places they make sense - and they are great there.
<lifeless> but - attractive nuisance everywhere else
<blr> lifeless: gather someone was a little over-enamoured with Cucumber?
<lifeless> blr: doctests are a very different thing to cucumber
<lifeless> blr: but if it had existed, they probably would have loved it :)
<blr> lifeless: the way they're using in LP is very similar
<blr> *used
<lifeless> there's none of the regex object lookup stuff
<lifeless> nor rich verifiers
<lifeless> because there's no DSL
<lifeless> so there's no place to fix the issues
<blr> right, I just mean in the 'BDD/tests as stories' sense
<blr> lifeless: I would be happier with them if they were less brittle I guess
<lifeless> yah
<blr> wgrant: fixed the setTargetDefault bug, should be ready for you to look at when you're free.
<blr> wgrant: one thing that could use improvement (although not certain how it works wrt the vocab), an invalid shortened_path returns "Invalid value" which could be nicer.
<wgrant> blr: "Invalid value" is the best Launchpad error.
<wgrant> blr_: ProductSeries:+index doesn't seem to work any more.
<blr_> wgrant: hrm, will have a look, thanks.
<blr_> ah, the git vocab context type
<blr_> wgrant: hmm, that vocabulary won't be used in the ProductSeries context of course, s
<wgrant> blr_: Shouldn't need to exist, no, since the field doesn't exist there.
<blr_> weird, tmux went bellyup... anyhow.
<blr_> wgrant: have pushed a fix, although it might not be the best approach.
<blr_> the vocabulary will be called in both contexts, so I've handled ProductSeries there.
<wgrant> blr_: Why is it being called when the widget isn't rendered there?
<blr_> the SetBranchForm schema is inherited
<wgrant> blr: It's common to customise self.field_names in setUpFields.
<wgrant> So you could only include default_vcs and the git bits in one case.
<blr> wgrant: right, can see that in some of the other views, that seems preferable to handling the unwanted context in the vocab!
<wgrant> blr: Yep. The only thing to worry about is fixing the update actions to only run the code when the relevant widgets aren't excluded.
<blr> wgrant: ok that should be looking happier now, thanks.
<blr> would be nice if the comment nav left a buffer at the top of the frame...
<cjwatson> wgrant: Are you happy with me requesting a prod git upgrade run?  I'd like to get turku in place again (it'll require slightly special instructions in the ticket, which I'll take care of).  It'll also pull in your recent turnip changes though.
<blr> evening cjwatson
<cjwatson> hiya
<cjwatson> will vanish shortly to put kids to bed :)
<blr> yep, see you in an hour
<cjwatson> blr: will you have a chance to finish off verbose-diff today?
<blr> cjwatson: will certainly try - just about finished with changes from wgrant's review
<blr> wgrant: regarding configure_codehosting on ProductBranchListingView, not certain I understand your comment - it isn't an override afaict.
<wgrant> cjwatson: Yep, turnip upgrade is fine.
<cjwatson> man I love being able to do entire meetings without making a noticeable dent in my battery
<wgrant> Heh
<cjwatson> wgrant: oh, BTW, I'm deliberately not landing avoid-pbuilder until the buildd upgrade happens, to avoid confusion with the recipe build that would follow a commit
<wgrant> cjwatson: Which confusion?
<cjwatson> perhaps we should change procedures there to copy from ~launchpad/ubuntu/buildd-staging instead of ~launchpad/ubuntu/ppa
<wgrant> Oh
<cjwatson> the current procedure has them copying from ~launchpad/ubuntu/ppa, which is the target of a recipe
<wgrant> IS confusion, right.
<cjwatson> that
<cjwatson> wgrant: I was thinking the other day about how we'd do git mirroring, incidentally; my thought is that we'd want to have a separate juju environment for pull/push workers (it wants rather different firewall rules from turnip, lots more outgoing), each of which keeps something like an LRU store of repositories which it reaps when it runs low on space; probably no point in sharing code with the existing codeimport stuff, it wants to be ...
<cjwatson> ... juju from the start and there's not a huge amount it can usefully share; haven't figured out how the dispatch would work yet; don't know how the existing codeimport stuff arranges to be sufficiently privileged
<wgrant> cjwatson: The existing codeimport stuff is valuable as a lesson in how not to do it.
<cjwatson> the job dispatch business is something we'd need to solve to jujuise bzr codeimport anyway, since we wouldn't want the workers talking to the DB
<wgrant> I was going to try avoiding having a duplicate copy of the branches.
<wgrant> The codeimport workers currently talk only to xmlrpc-private and librarian upload.
<cjwatson> The only way I can see to do that is to allow turnip outbound access; or perhaps a custom twisted thing to proxy git receive-pack/upload-pack
<wgrant> I believe a bridge may be feasible.
<cjwatson> Yeah, might even be buildable out of turnip code perhaps
<cjwatson> It has most of the right pieces for it
<wgrant> It just needs to handle the have/want negotiation between the remote upload-pack and the local receive-pack.
<wgrant> That was always my preferred solution, but I don't have a proof of concept.
<mwhudson> yes, don't do what the codeimport stuff does
<mwhudson> figuring out how to let launchpad services have privileged access to branches would be nice too...
<wgrant> You mean other than unauthenticated-but-firewalled read-only slow HTTP? :)
<mwhudson> ideally
<mwhudson> that's codebrowse you're talking about there?
<wgrant> All internal services access Bazaar branches using the same mechanism.
<wgrant> Scanning, diff generation, etc.
<mwhudson> oh right, yes
<mwhudson> what i meant, as i'm sure you know, is things like recipes for private branches
<mwhudson> and also having code imports just being able to push to the branch directly
<wgrant> Yep.
<mwhudson> (but not having code import workers having the ability to just push to any branch...)
<wgrant> Now that we have HTTPS git repo access that's probably more reasonable.
<wgrant> No need for SSH keys. We can just create temporary tokens.
<cjwatson> We'd need auth tokens
<cjwatson> Yeah
<wgrant> Similar to the librarian's TLTs, except that they can be revoked once the job is done.
<mwhudson> i half implemented this for the codehosting ssh server once
#launchpad-dev 2015-06-25
<blr> huh found a bug
<wgrant> blr: In what?
<blr> _validateImportExternal, validate_import_url was only called with one argument
<wgrant> The tests didn't catch that?
<blr> not until now apparently...
<blr> just fixing up all the doctests I've broken with +setbranch -> +configure-code
<wgrant> There may indeed be a few.
<blr> cjwatson: the str(line) was needed as line is a Context/RemoveLine
<blr> cjwatson: have a working failing test, you were correct, thanks
<blr> "working failing test" ...
<wgrant> blr: Beware that there are in fact more test failures than buildbot reports. Search for "failure:" in the run's stdio to find them all.
<wgrant> blr: The subunit stream has been corrupted.
<blr> ah some unittests too, thanks
<blr> hmm, any idea what's going on with the product-registry doctest?
<wgrant> Which?
<blr> xx-project-registry.txt, the "..." matcher doesn't appear to be matching "api.luanchpad.dev/beta"
<wgrant> Oh, you put too much faith in doctests.
<wgrant> All ... will show as mismatches in the difference output.
<wgrant> You need to ignore them and find the real different; in this case that inferred_vcs was added.
<blr> ah of course, thanks
<blr> wgrant: should I be concerned that we're bumping query limits?
<blr> fairly rhetorical question.. but something I should think about?
<wgrant> blr: I'm not sure why it would be doing that there. Is something invoking inferred_vcs, perhaps? Should it be?
<blr> wgrant: slightly confused that it blew up on blueprints
<blr> also, what an excellent idea, wasn't aware of these tests.
<blr> have fixed everything, will --testfix after dinner
<blr> wgrant: could you have a look over this please when convenient https://code.launchpad.net/~blr/launchpad/ui-project-setbranch/+merge/262942
<wgrant> blr: Did you work out what the extra queries were?
<wgrant> Oh, it'll be inferred_vcs in the embedded JSON representation, of course.
<wgrant> blr: r=me, thanks.
<blr> yep
<blr> thanks
<blr> missed one -_-
<blr> wgrant: will just top-approve this one, hope that's okay (small fix)
<wgrant> blr: Yep, trivial testfixes are fine to self-approve unless they're stupid :)
<cjwatson> wgrant: Hm, https://launchpad.net/ubuntu/+source/libmoox-cmd-perl/0.013-2/+build/7541385 should have been a dep-wait, surely?
<cjwatson> wgrant: Oh, no, never mind, that's uninstallability more than one level deep
<cjwatson> wgrant: This is maybe a little unfortunate, because I think it really is just that one.  Probably an sbuild bug in that it should be using "apt-get build-dep".
<cjwatson> Yeah, everything's fine in that build-dependency set apart from the version restriction
<cjwatson> Tempting to do http://paste.ubuntu.com/11772424/
<wgrant> cjwatson: Will that reliably cope with alternatives?
<wgrant> There are some complexities there.
<cjwatson> build-dep should in general cope better than install.  The only reason I didn't do it in the first place in that code was that I was being conservative when fixing up cross handling.
<cjwatson> But it certainly makes sense to check that.
<cjwatson> Hm, it's perhaps possible that it would break the --no-resolve-alternatives case ...
 * cjwatson leaves an XXX comment for himself to check that later
<cjwatson> wgrant: https://code.launchpad.net/~cjwatson/launchpad/side-by-side-diff/+merge/262768 should be worth another look now.
<blr> morning
<blr> cjwatson: if you're about by any chance, added a test and fixed that bug.
<bigjools> wgrant: do you know if there's a way to influence which ssh key is used for sftp dput uploads? Unless there's a sneaky thing in bzrlib I can't see one :(
<lifeless> bigjools: just make sure the right one is in your ssh agent
<bigjools> lifeless: what if there's no agent?
<lifeless> bigjools: then as long as its in the search path of default keys
<lifeless> bigjools: or you can put it in the ssh command as an explicit parameter
<lifeless> bigjools: (see bzr help config)
<bigjools> lifeless: there's not much configuration you can do for dput though
<bigjools> the plugin does:         t = bzrlib.transport.get_transport("sftp://%s@%s/%s" % (login, fqdn, incoming))
<lifeless> bigjools: yes, that doesn't invalidate what I said
<bigjools> lifeless: there's no ssh command though and I can't see how to select a specific key
<lifeless> digging
<bigjools> lifeless: It looks like the bzrlib sftp transport ignores the .ssh/config but it should not according to what I read elsewhere :/
<lifeless> bigjools: if its using paramiko rather than openssh that could happen
<bigjools> I believe it's paramiko, yes
<lifeless> well
<lifeless> we always use some bits of paramiko
<lifeless> but
<lifeless> its whether the channel setup is paramiko or ssh
<lifeless> bigjools: see bzr help authentication
<lifeless> bigjools: it might help, I haven't re-read it all
<bigjools> cheers, reading
<bigjools> looks great, but nothing about keys :(
<lifeless> ahha
<lifeless> BZR_SSH
<lifeless> BZR_SSH             Path to SSH client, or one of paramiko, openssh, sshcorp,
<lifeless>                     plink or lsh.
<lifeless> try setting that to openssh
<lifeless> it may cause an error but then you'll know why paramiko was being used
<bigjools> aha!
<lifeless> (bzr help env-variables)
<bigjools> I found the bzrlib code that only uses id_dsa or id_rsa :(
<lifeless> yeah, thats in the paramiko channel setup
<lifeless> which TBH you don't want to use anyway
<bigjools> just trying it out, will let you know shortly
<bigjools> and it worked!  thanks a million lifeless
<lifeless> de nada
<bigjools> cider++
#launchpad-dev 2015-06-26
<wgrant> cjwatson: Are you sure https://bugs.launchpad.net/launchpad-buildd/+bug/1468755 is a regression?
<mup> Bug #1468755: Packages with unsatisfied versioned BDs don't go into depwait <regression> <launchpad-buildd:Triaged> <https://launchpad.net/bugs/1468755>
<wgrant> I didn't know the context when we discussed it yesterday.
<wgrant> I'm pretty sure that case would always have failed, unless I'm missing something.
#launchpad-dev 2015-06-28
<lifeless> wgrant: hmm, I can't find the oauth connect bug that IIRC I opened on LP.... ring any bells ?
<wgrant> lifeless: OAuth Connect isn't a thing.
<lifeless> ah yes bad search terms
<wgrant> I'm not sure you filed a bug about the OAuth thing you ran into, though.
<lifeless> and found it
<wgrant> Oh, where?
<lifeless> https://bugs.launchpad.net/launchpad/+bug/1420348
<mup> Bug #1420348: Support OpenID Connect <Launchpad itself:Expired> <https://launchpad.net/bugs/1420348>
<lifeless> I remembered your question and wanted to sit down and think abou tit
<wgrant> Oh, I thought you meant the OAuth Haskell thing you had trouble with.
<lifeless> no, my patches were merged into a release ages ago
<lifeless> I've finally gotten back to that side project in fact
<lifeless> but thats separate
<lifeless> wgrant: does SSO support OAuth 2 ?
<wgrant> lifeless: Not last I knew, but I haven't checked in a while.
<lifeless> ok, commented :0
<blr_> morning
<wgrant> Morning blr
<blr> hi wgrant, how was your weekend?
<wgrant> blr: Pretty good. You?
<blr> good, have family from out of town here atm, so busy... glad i can hide in my office today :)
<blr> I see your pygit2 issue still hasn't had a response heh
<wgrant> Indeed, but no rush.
<cjwatson> wgrant: I'm pretty sure that dep-waited before, yes, because the old sbuild noticed the unsatisfiable version after installing and we picked up its strange output.  TBH I suspect the right answer to this is to suck up the work of hooking in dose-builddebcheck.
<wgrant> cjwatson: Oh, apt pick an installable older version? Hmm.
<wgrant> cjwatson: Do you have a plan for dose-builddebcheck? It's rather messier for us as we have an awful lot of archives.
<wgrant> (and then there are questions like bootstrap archives)
#launchpad-dev 2016-06-27
<xnox> cjwatson, wgrant - can launchpad be used as a generic GPG signing service? Specifically looking for something to generate, keep keys, sign things, and return signed objects....
<xnox> and/or is there any other best practices you can point me at building a signing webservice?
<xnox> (internal or public)
<cjwatson> xnox: Launchpad cannot in general.  I don't have advice to point you at although I'm sure that exists somewhere; perhaps "SoftHSM" or similar would be a useful search term.
#launchpad-dev 2017-06-26
<xnox> I guess this will never start? https://launchpad.net/~cloudware/+livefs/ubuntu/artful/cpc/+build/102259
<xnox> due to missmatch of project?
<cjwatson> Can't tell because private
<cjwatson> Is it perhaps a powerpc or s390x build that you got webops to enable?
<cjwatson> If so did you remember to ask them to devirtualise it as well?  They usually forget if not asked explicitly, IME
<xnox> cjwatson, that!
 * xnox feels like an idiot, given this happens over and over again, and i new about it.
#launchpad-dev 2017-06-27
<xnox> cjwatson, is there something special that needs doing to enable livefs builds for a user? https://launchpad.net/~xnox/+livefs/ubuntu/artful/ubuntu-server-live/+build/102353 should be public and is not building =(
 * xnox is trying to hack up a way to spin up subiquity s390x livefs
<wgrant> xnox: That livefs is not configured as non-virt.
<xnox> ah
<xnox> require_virtualized
<xnox> (writeable)
<xnox> Require virtualized builders
<xnox> Only build this live filesystem image on virtual builders.
<cjwatson> Right, exactly the same thing I mentioned last night :)
<xnox> i did not realise it's a setting on the livefs, rather than the PPA =)
<xnox> now i see require_virtualized in the livefs object in the api docs
<cjwatson> PPAs have it as well, but the livefs setting is independent because a livefs is built from a PPA, not into a PPA
<cjwatson> (or from an archive, in general)
<xnox> right.
<xnox> cjwatson, please set require_virtualized to False on https://api.launchpad.net/devel/~xnox/+livefs/ubuntu/artful/ubuntu-server-live ? it's for me to build s390x subiquity images
<xnox> i will build things with my nonvirt ppa
<xnox> which has s390x enabled.
<xnox> also, i thought scalingstack has s390x now, such that slowly s390x kvm builders can be added.
<wgrant> xnox: Not yet; we're still one GRE tunnel from having a working cloud there.
<xnox> (and then decommissioning z/VMs and getting more kvm builds)
<wgrant> And there'll be nothing slow about that migration :)
<xnox> right =) i see
<wgrant> I expect it to take all of a day, and 23h30 of that day will be an abundance of caution.
<xnox> love it "working" cloud. Somehow it seems the bar is high for wgrant to bless something as "working"
<wgrant> The cloud can spawn instances. They just can't talk to the outside world yet, so they're not very useful.
<cjwatson> xnox: Done, though you'll probably have to cancel the pending build and re-request it.
<xnox> ack, thank you!
<xnox> win \o/ it's building. not just have to make it work.... =)
#launchpad-dev 2017-06-28
<xnox> wgrant, cjwatson: is it true that s390x livefs launchpad builders cannot access the snap store? Or e.g. more proxies are needed?
<xnox> I'm seeing:
<xnox> Fetching snap "core"
<xnox> error: cannot find snap "core": Get https://search.apps.ubuntu.com/api/v1/snaps/details/core?channel=stable&fields=anon_download_url%2Carchitecture%2Cchannel%2Cdownload_sha3_384%2Csummary%2Cdescription%2Cdeltas%2Cbinary_filesize%2Cdownload_url%2Cepoch%2Cicon_url%2Clast_updated%2Cpackage_name%2Cprices%2Cpublisher%2Cratings_average%2Crevision%2Cscreenshot_urls%2Csnap_id%2Csupport_url%2Ccontact%2Ctitle%2Ccontent%2Cversion%2Corigin%2Cdeveloper_id%2Cprivat
<xnox> 2Cconfinement%2Cchannel_maps_list: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
<xnox> E: config/hooks/032-installer-squashfs.binary failed (exit non-zero). You should check for errors.
<xnox> but the core snap does exist on s390x in the stable channel
<xnox> the command run there is:
<cjwatson> xnox: It's not intended, although it may currently happen to be true
<xnox> + sudo SNAPPY_STORE_NO_CDN=1 snap download core
<xnox> which works on my dev box.
<xnox> cjwatson, ok. it would be nice to check that with the virt builders. I think I will only need these bits to work, only after we have proper builders.
<cjwatson> xnox: File an RT ticket - it'd be a firewall change, and I don't know exactly what route the s390x builders take there
<xnox> ack.
<cjwatson> xnox: We fixed the virt builders for that ages ago, so no check needed
<cjwatson> xnox: https://rt.admin.canonical.com/Ticket/Display.html?id=98451
<cjwatson> er, ish
<cjwatson> xnox: However
<cjwatson> xnox: Apparently bos02 is doing pretty well so consider just waiting until we have virt s390x builders instead; that also avoids the question of just what evil might be installed off the snap store onto a non-virtualised builder and persist forever
<xnox> yeah, that.
<cjwatson> (search.apps.ubuntu.com / api.snapcraft.io is one thing, but the CDN is another)
<xnox> we do bypass CDN in the livefs build, on purpose.
<cjwatson> OK, but the issue of arbitrary not-very-controlled code on non-virt builders is still open
<xnox> true
#launchpad-dev 2017-06-29
<cjwatson> wgrant: Job leases confuse me.  Do you know of a particular reason why queueing a job shouldn't just release the lease?  AFAICT retry_delay should be sufficient to arrange for it to be queued a reasonable time in the future, and having the lease still held just confuses things.
<wgrant> cjwatson: retry_delay is only for celery, isn't it?
<wgrant> I agree it's confusing, but it's not *just* confusing.
<cjwatson> Oh, does nothing else honour scheduled_start?  Hmm.
<wgrant> Oh, it's possible
<cjwatson> Job.ready_job checks scheduled_start
<cjwatson> ready_jobs
<cjwatson> So surely a job won't show up in iterReady() until the retry delay has elapsed.
<wgrant> You are probably not wrong.
<wgrant> It was all slightly strange, and I ripped out some strangeness a few years ago, but not all of it.
<cjwatson> (I'm trying to fix the weird snap store upload retry behaviour.  I have it mostly fixed, but it's a bit of a hairball.)
#launchpad-dev 2017-06-30
<sil2100> o/
<sil2100> What is the best/easiest way to get the lp login identifier of the person the user is currently logged as through Launchpad API?
<wgrant> sil2100: There's no direct API for that. Can you explain your use case?
<sil2100> wgrant: I'm just improving some sru-review tooling and I'd like to do 'something special' when a bug task is assigned to the person running the review tool
<wgrant> sil2100: Hm, why does that require the OpenID identifier?
<sil2100> wgrant: not the openID identifier, just the user LP id, e.g. the username I meant
<sil2100> Maybe I phrased this wrongly in my question
<wgrant> sil2100: Oh. lp.me.name
<sil2100> wgrant: exactly what I need, thank you!
<wgrant> sil2100: np. Consider using lp.me.self_link and comparing it to bug_task.assignee_link, rather than doing extra roundtrips to compare the username directly.
<sil2100> Yeah, will do, the lp.me was what I needed, completely forgot about it
<wgrant> sil2100: That's just launchpadlib sugar around lp.load('/~') or lp.load('/people/+me') -- all the paths match the web UI.
#launchpad-dev 2018-06-25
<jelmer> cjwatson: yep
<xnox> cjwatson, wgrant is there anything that could be done about https://bugs.launchpad.net/launchpad/+bug/1727496 ? it hinders us processing ibm-java80 updates in a timely manner, and requires a bit of manual tracking of "upload ibm-java80, yet update ibm-java71 bugs"
<mup> Bug #1727496: can't reassign bug to ibm-java80 package <Launchpad itself:New> <https://launchpad.net/bugs/1727496>
<cjwatson> xnox: It's probably a regression from http://blog.launchpad.net/beta/beta-test-new-package-picker, but you should be able to work around it using the API
<xnox> cjwatson, ooooh, interesting. I will try the API thanks!
<xnox> if i can reassign it once these bugs come in, it should then continue to work fine for everybody else.
<xnox> *via the API
<cjwatson> I see no reason why you should believe that
<cjwatson> Its publication status won't change by you reassigning some bugs to it, so I wouldn't expect the picker's behaviour to change
<xnox> cjwatson, right, but people can find the bug report under the right name; and it will be assigned to the right package name; and nobody will get confused anymore
<xnox> cjwatson, unless i go and break the bug via manipulation of the bug tasks, that is =/
<xnox> cjwatson, https://launchpad.net/ubuntu-z-systems/+bug/1775641 no longer loads via webui
<mup> Bug #1775641: [Comm] IBM JDK 8.0.5.16 integration into Ubuntu <architecture-s39064> <bugnameltc-168695> <severity-high> <targetmilestone-inin---> <Ubuntu on IBM z Systems:In Progress by ubuntu-archive> <ibm-java80 (Ubuntu):New for sil2100> <ibm-java80 (Ubuntu Xenial):Fix Committed> <ibm-java80
<mup> (Ubuntu Bionic):New> <https://launchpad.net/bugs/1775641>
<cjwatson> works for me but presumably after you deleted some tasks
<cjwatson> xnox: can you leave it alone for a bit and I'll see if I can get them in place?  I think you must have broken conjoined tasks
<xnox> cjwatson, yes, i've deleted all the tasks, and readded ubuntu-z-systems project task, and saved everything.
<xnox> there were duplicate tasks/targets, multiple.
<xnox> it loads now.
<xnox> (and saved all the tasks & bug, after deleting all the duplicate tasks)
<cjwatson> in oopsing state again, debugging
<cjwatson> ah, hmm - sort of the same bug, because it can't initialise the edit form
<xnox> ð i shall hang my head in shame for totally breaking it now
<cjwatson> that is quite adjectival
<xnox> protip since forever - try thing on staging first...
<cjwatson> shall I put back the ibm-java71 tasks for now?
<cjwatson> this is going to need some work on the webapp
<xnox> yes please
<xnox> if they will still load.
<cjwatson> xnox: done; you'll need to put back the status of the ubuntu-z-systems task (I think it was In Progress / Medium)
<cjwatson> and assigned to ~ubuntu-archive
<xnox> tah
#launchpad-dev 2018-06-27
<cjwatson> wgrant: Somewhere there is existing code that produces a modified field for some page or other for admins, because they have far too many options to use the normal vocabulary sensibly.  Does that ring a bell and if so do you remember where it is?
<cjwatson> My grepping isn't turning up anything.
<wgrant> cjwatson: Project owner maybe?
<wgrant> No, branch owner
<wgrant> I think
<cjwatson> Ah yeah, could be
<wgrant> cjwatson: CodeEditOwnerMixin
<cjwatson> Though maybe this isn't a problem in the case at hand.  For some reason in https://oops.canonical.com/?oopsid=OOPS-bdc9d2c557980f1cad47c5bc49c51a86 BuildableDistroSeries.findSeries is deathspiralling (fair enough) but target_ppas_vocabulary isn't (weird)
<cjwatson> Fixing the former is easy enough so maybe I'll just try that and only fix the latter if it's demonstrably a problem
<wgrant> cjwatson: Does it get to the point where it would even materialise the latter?
<cjwatson> Yes - make_archive_vocabulary does that
<wgrant> Hmm
<cjwatson> Iterates through all the archives and does archive.reference etc.
<cjwatson> four million people and <100000 archives though, so maybe it gets away with it somehow
<cjwatson> I'd have expected materialising 100000 archive rows to be worse than 300ms though.
<cjwatson> And the resulting page must be terrible ...
<wgrant> It is weird, unless set is taking forever for some reason on a set of 100000 identical Storm objects.
<wgrant> Which doesn't seem entirely likely.
<cjwatson> And anyway that would explain findSeries being slower, which I already thought was clear enough, but not target_ppas_vocabulary being mysteriously fast.
<cjwatson> Oh well, maybe I'll see if an OOPS with the first thing fixed shows any big differences.
<cjwatson> If it does then there's still room to bulkify the vocabulary construction for people like us who merely have a couple of thousand usable PPAs, and do something like CodeEditOwnerMixin for admins.
#launchpad-dev 2020-06-22
<tomwardill> Convert QuestionMessage to Storm: https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/386166
<SpecialK|Canon> nice
<SpecialK|Canon> I mean
<SpecialK|Canon> â¡
<tomwardill> unsure if the list()ification is the appropriate technique there
<tomwardill> I appear to be reading the debian packaging tutorial
<tomwardill> send Monday back, I don't want it any more
<ilasc> ð
<pappacena> I have kind of small MP to allow users to edit their code import URLs here: https://code.launchpad.net/~pappacena/launchpad/+git/launchpad/+merge/384586
<pappacena> If someone could find some time to review that... it shouldn't be complicated, since the details were already discussed in the MP page itself.
<tomwardill> pappacena: will look in a mo, just arguing with insurance companies
 * pappacena ð
<pappacena> Good luck!
<tomwardill> pappacena: +2
<tomwardill> 2?
<tomwardill> errr, 1
<pappacena> I guess +2 is even better... hehe
<roadmr> insurance????????????
<roadmr> yuck!
<tomwardill> yeah, had a whoopsie in my van
<tomwardill> now the van is owned by the insurance company
<tomwardill> I'd quite like the rest of the insides of it back, this is apparently somewhat novel.
<roadmr> wow.
#launchpad-dev 2020-06-23
<tomwardill> hmm, there's some doc test fallout from the code import changes, unsure as to correct way to fix
<ilasc> yep, saw that
<ilasc> and can't seem to reproduce locally
<ilasc> ok, got it locally
<ilasc> meaning reproduced
<wgrant> The diff on that MP is very weird
<ilasc> tomwardill: do you want me to look at it or are you ?
<tomwardill> `bin/test -vvct lib/lp/code/doc/codeimport.txt` on master will do it
<wgrant>      <require
<wgrant> +       permission="launchpad.Edit"
<wgrant> +       set_attributes="url"/>
<wgrant> url has a separate setter method AFAIK which performs extra checks
<tomwardill> wgrant: the doctests imply it does, but I can't find any reference to it actuallye xisting
<wgrant> Oh, interesting.
<tomwardill> there is `updateFromData`, but there's no setters for the attributes
<wgrant> Oh, yeah, that's what I mean
<wgrant> And what it means.
<wgrant> Not as in a property.setter, but a general accessor.
<wgrant> The fix here is to remove that extra security declaration and fix any test fallout from that.
<tomwardill> poking now
<tomwardill> wgrant, ilasc: Doing this seemed a sensible way forward: https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/386241
<tomwardill> otherwise updateFromData grows a bunch of 'if' checks for permissions on certain attributes.
<tomwardill> thanks ilasc :) wgrant, can you just check we've not missed something easier with the zcml? I get suspicious of xml, it's sneaky.
<wgrant> Needs more gaml
<wgrant> tomwardill: Looks fine I think
<tomwardill> ta, landing
<wgrant> Only objection is that updateFromData could use updateURL, but that's hard with how the former is currently implemented
<wgrant> So this way around is good
<tomwardill> yeah, it's a little too wrapped up in the notification checks to extract easily
<tomwardill> I bet me landing the QuestionMessage changes breaks a bunch too
<ilasc> ð
<ilasc> I guess we'll find out
<tomwardill> sigh, yep
<wgrant> Easy ones, at least.
<tomwardill> yeah, got a branch to fix, just waiting to see if there's any more
<tomwardill> https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/386248 counting is hard
<ilasc> lol
<pappacena> Thanks for fixing the doctest, folks. Sorry about that! I didn't see the build bot failure.
<tomwardill> no worries :)
<tomwardill> hmm, this LXD transcription is going to have some 'interesting' features here...
<SpecialK|Canon> Obligatory reminder to please signpost all the known dragons...
<tomwardill> it's okay, I've named them all Errol
<SpecialK|Canon> I really must read Discworld some day
<tomwardill> start at Guard Guards.
<tomwardill> (this is a semi-controversial opinion)
<tomwardill> pylxd does not make things easy
<tomwardill> to the point where actually just shelling out and using bash is a lot easier
 * tomwardill -> EOD for bicycle club
<SpecialK|Canon> That's disappointing :(
<SpecialK|Canon> Er, the pylxd thing not the bicycle club!
<SpecialK|Canon> Happy cycling!
#launchpad-dev 2020-06-24
<ilasc> pappacena: MP 386286 looks good to me but definitely needs input from wgrant
 * pappacena thanks! :D
#launchpad-dev 2020-06-26
<wgrant> We should probably feature flag out the search/create OCI project links on Product:+index until the UI is a bit more polished.
<wgrant> e.g. all project pages now have "Search for OCI Projects" link at the top right, which for the great majority of projects is going to return nothing
<wgrant> Product:+new-import is timing out very weirdly on qastaging.
