[00:06] <timrc> I'm working on an app that authenticates users with lp/openid and I'd like to use their team memberships for context.  I must be missing something because using OPENID_LAUNCHPAD_TEAMS_MAPPING_AUTO = True in conjunction w/ https://login.launchpad.net/ does not seem to do anything
[00:06] <timrc> Oh, and one important detail, I'm using django and the django-openid-auth library
[00:11] <lifeless> timrc: I think you need to be whitelisted to query that via openid
[00:11] <lifeless> to prevent harvesting; IMBW
[00:11] <timrc> lifeless: thanks! do you know who I would talk too?
[00:11] <lifeless> timrc: anyhow, separately, don't use login.launchpad.net - login.ubuntu.com *is* login.launchpad.net
[00:12] <lifeless> https://forms.canonical.com/lp-login-support/
[00:12] <timrc> lifeless: right, thanks and thanks again
[00:12] <lifeless> from the bottom of https://login.launchpad.net/
[00:12] <lifeless> bach
[00:12] <lifeless> https://login.ubuntu.com/
[00:12] <lifeless> https://forms.canonical.com/sso-support/ <- that one
[00:14] <timrc> lifeless: great, thank you
[00:14] <lifeless> jml: around?
[00:40] <pcjc2> lifeless - that SQL query
[00:41] <pcjc2> I wrote it out indented, was slightly perturbed to see lack of brackets between OR and AND clauses in the bug status selection portion
[00:41] <pcjc2> might be correct, but the operator ordering isn't totally clear
[00:42] <pcjc2> actually, on second reading, probably OK - I must avoid adding too many () for the sake of not having to remember operator ordering!
[00:44] <pcjc2> I trust the postgreql engine is smart enough not to execute the Auth lookup part of the query in the case where Bug.private = FALSE
[00:44] <pcjc2>          EXISTS (SELECT BugSubscription.bug FROM BugSubscription, TeamParticipation
[00:44] <pcjc2>                         WHERE TeamParticipation.person = 882987 AND
[00:44] <pcjc2>                               BugSubscription.person = TeamParticipation.team AND
[00:44] <pcjc2>                               BugSubscription.bug = Bug.id)
[00:44] <pcjc2> could take a while if executed for every bug it has to consider in the list
[00:46] <lifeless> thats a correlated subquery yes
[00:48] <pcjc2> I used to speak SQL at one point (A-Level computing) ;)
[00:48] <pcjc2> but not so much any more - its mostly read-only
[00:49] <pcjc2> I'm trying to run it on my local instance (although doesn't have such a big DB ;))
[00:49] <lifeless> hey if you've reformattted it to work
[00:49] <lifeless> paste that to me in pastebin or something
[00:50] <pcjc2> not reformatted, just entered in psql
[00:50] <pcjc2> reindented it so I could read it
[00:50] <pcjc2> doesn't get any hits on my DB - so I suspect I didn't munge the product IDs properly to reflect my local instance
[00:55] <pcjc2> BugTask.bug NOT IN (SELECT bug FROM BugTag) AND
[00:56] <pcjc2> what about having NOT EXISTS (SELECT bug from BugTag where bug = Bug.id)
[00:56] <pcjc2> so you don't search the entire tag list for every bug in the DB for every bug we query?
[00:57] <pcjc2> or perhaps "BugTask.bug NOT IN (SELECT bug FROM BugTag WHERE bug = Bug.id)"
[00:57] <lifeless> pcjc2: there were \\n in the thing I pasted
[00:57] <lifeless> if you can paste me something ready to run
[00:57] <lifeless> then I can do something interesting
[00:57] <pcjc2> ok - will try
[00:57] <pcjc2> I deleted the \\n and indented differenly, but never mind
[00:58] <pcjc2> LP ought to grow a "no wrap" mode for some comments IMO
[01:00] <pcjc2> http://ubuntu.pastebin.com/DVg1NzYm
[01:00] <pcjc2> try that
[01:00] <pcjc2> Just adds a "WHERE bug = Bug.id" to the tag sub-query
[01:01] <pcjc2> I'm curious what bit of code translates the tag query the user enters into that SQL though - other than the missing WHERE clause, it looks pretty efficient
[01:02] <pcjc2> (of course.. I'm not certain that adding the WHERE clause will help - it should in theory be able to do just as good a search without it, since it is filtered by that afterwards)
[01:03] <thumper> lifeless: hi
[01:03] <thumper> lifeless: do we have regex string matchers yet?
[01:03] <thumper> lifeless: just wondering if someone else has written them yet
[01:04] <lifeless> thumper: in testtools trunk I think
[01:04] <thumper> :(
[01:05] <pcjc2> lifeless: are you able to run that SQL on the same DB as a real server? (If not, we need different owner IDs and product Ids)
[01:06] <pcjc2> Since the query is ~instant on my machine it probably needs testing on a large DB to reproduce the huge exec time
[01:14] <pcjc2> if that helps (not that I probably need to point you at this ;)), the SQL in question comes from lib/lp/bugs/model/bugtask.py ~ line 1369
[01:15] <pcjc2> Sorry, line 1422
[01:15] <pcjc2>             exclude_clause = "SELECT bug FROM BugTag"
[01:15] <lifeless> ok, 0.3 seconds
[01:15] <lifeless> hmm
[01:15] <lifeless> we don't have your data in prod yet
[01:15] <lifeless> hang on
[01:16] <pcjc2> and 17 when executed before?
[01:18] <spm> lifeless: https://pastebin.canonical.com/41738/
[01:19] <spm> 2nd run https://pastebin.canonical.com/41739/
[01:19] <pcjc2> s[is this related to what I'm looking at?
[01:19] <lifeless> spm: thanks; what does it return without the explain analyse - whats the found count ?
[01:20] <pcjc2> spm: ... (^___)
[01:20] <lifeless> pcjc2: yes, its your query analyzed
[01:20] <pcjc2> don't have privs to look at that
[01:20] <spm> o rows
[01:20] <lifeless> ok, thats very interesting
[01:20] <spm> that'd be 0. not o.
[01:20] <lifeless> pcjc2: did you add  braces perhaps ?
[01:20] <pcjc2> That comes from the fact it was a stupid query for our bug-set
[01:21] <pcjc2> ALL the bugs are tagged with something to determine which sourceforge tracker they came from
[01:21] <lifeless> pcjc2: does it still oops if you try via the web UI ?
[01:21] <pcjc2> I only realised this when I tried to run it locally
[01:21]  * pcjc2 checks
[01:22] <pcjc2> Yes, it does
[01:22] <pcjc2> (Error ID: OOPS-1837O107)
[01:22] <pcjc2> Any chance I can see those pastebin postings?
[01:23] <lifeless> ah yes
[01:23] <lifeless> http://pastebin.com/732mWULr
[01:26] <pcjc2> most of the parts look like they were never executed
[01:26] <pcjc2> Is that on the production database?
[01:28] <lifeless> yes
[01:28] <pcjc2> so the same thing the OOPs said took 17 seconds, executed in 0.369ms ??
[01:28] <lifeless> unless there was a transcription error
[01:29] <pcjc2> doesn't show me in the dump what product IDs were used
[01:29] <lifeless> flacoste: I've removed the 'edge' series to avoid confusing folk
[01:33] <lifeless> spm: again please - http://pastebin.com/raw.php?i=3S3RKCSY
[01:34] <lifeless> spm: two by explain analyze and one for the result itself
[01:36] <lifeless> pcjc2: that will be an hour or so
[01:36] <pcjc2> execution time, or just fitting the job in?
[01:36] <lifeless> getting an spm timeslice
[01:37] <pcjc2> spm is a LOSA ?
[01:39] <lifeless> yes
[01:39] <pcjc2> just for scale - what sort of size data-centre is all this running on?
[01:40] <lifeless> data centres
[01:40] <wgrant> The DB servers are amusing.
[01:40] <lifeless> pcjc2: primary db server is 16 cores, 128GB of ram, db is ~250GB on disk
[01:41] <lifeless> pcjc2: we have 2 active replicas with 8 cores eachm same ram, same db size on disk
[01:41] <pcjc2> I assumed some kind of DB cluster
[01:41] <pcjc2> so queries are load-balanced between all 3?
[01:41] <lifeless> appservers direct read only queries to replicas when possible
[01:41] <lifeless> writes go to the primary
[01:42] <pcjc2> was just about to ask that
[01:42] <pcjc2> does it gaurantee in-order processing?
[01:43] <lifeless> no
[01:43] <pcjc2> so if you get a write to the primary, followed by a read at some point later (from the same app?), the write is going to be replicated and affect the read?
[01:43] <lifeless> we so, uhm 4million page hits a day
[01:43] <lifeless> if we serialised we'd be dead in the water
[01:43] <wgrant> pcjc2: The DB is determined largely by the request type.
[01:43] <lifeless> a write to the primary and then a read from a slave may read the old data
[01:43] <wgrant> pcjc2: POSTs go to the master, most GETs to the slave.
[01:43] <wgrant> pcjc2: So within a request it should be consistent.
[01:44] <wgrant> Across requests, possibly not.
[01:44] <pcjc2> cool - this is computing on a scale larger than I ever knew before (writing MS Access databases ;))
[01:44] <pcjc2> lifeless: that first analyze execute you posted the results for..
[01:45] <pcjc2> was that the SQL query which LP executed, the one from the OOPS, or the modified one?
[01:45] <lifeless> all the one you gave me
[01:45]  * pcjc2 checks i didn't screw it up
[01:46] <lifeless> I've regenerated - its the new one I'm asking spm to run
[01:46] <pcjc2> I screwed it up...
[01:46] <pcjc2> The one I pasted you had product IDs replaced as I was running it on my local DB
[01:47] <lifeless> I suspected it might
[01:47] <pcjc2> I saw the IDs ok in the new one you asked to execute
[01:48] <pcjc2> how does security work here - I presume at some level, it is because the LOSAs know who you are and won't ask them to execute a DROP *;
[01:48] <pcjc2> And presumably they scan the SQL and keep an eye for any nasty side-effects possible?
[01:48] <lifeless> its because a) they know who I am and b) they will read the sql
[01:48] <lifeless> I'm authed to freenode, etc.
[01:49] <pcjc2> I recall hearing the LOSAs don't like executing SQL much
[01:49] <pcjc2> (Or certainly - not tested and approved SQL)
[01:49] <lifeless> thats nuanced
[01:49] <lifeless> mutating sql we require a bug describing how we can avoid ever running the sql again
[01:50] <lifeless> read only sql we only do as part of diagnosing faults
[01:50] <lifeless> and every such request is an interrupt which reduces project work
[01:50] <lifeless> so its moderately important to us not to interrupt them casually
[01:50] <pcjc2> so if there was some manual fixup required which could be done in a few SQL statements.. someone would have to file a bug and decide if LP needed to grow internal support for that feature etc..
[01:51] <lifeless> pcjc2: yes
[01:51] <lifeless> pcjc2: and we'd have an incident report
[01:51] <lifeless> our schema is extremely complexx
[01:51] <pcjc2> noted ;)
[01:51] <lifeless> the chance of screwing something up on even simple sql is nontrivial
[01:51] <pcjc2> I've been reading bits of it for amusements sake
[01:53] <pcjc2> When I run ANALYSE EXECUTE here I get      ->  Index Scan using bugtag__bug__idx on bugtag  (cost=0.00..8.27 rows=1 width=4) (actual time=0.003..0.003 rows=1 loops=267) for the tag lookup
[01:53] <lifeless> that will be affected by the stats which affect th eplanner
[01:53] <lifeless> also
[01:53] <lifeless> I wouldn't obsess on the tag component
[01:54] <lifeless> we need data
[01:54] <pcjc2> trye
[01:54] <pcjc2> But without the SELECT, there is rows=33445 loops=1
[01:54] <pcjc2> So I'm guessing that it will show a difference in the query you analyse too
[01:55] <pcjc2> Is spm going to run ANALYSE EXECUTE  on the original SQL LP emitted?
[01:57] <pcjc2> I'm going to have to call it a night - 2AM here - will read the bug report tomorrow to see if there was any development
[01:57] <pcjc2> (Please leave a comment if there is marked difference in analysis results with the tag part changed)
[01:57] <pcjc2> Btw.. in case you didn't hear it enough - Launchpad is AWESOME
[01:57] <lifeless> thanks :)
[01:58] <pcjc2> Moving from Sourceforge to LP with our bugs has TOTALLY reinvigorated our two projects
[01:58] <pcjc2> Contributors are coming out of the woodwork with patches, participating in review, reworking their patches
[01:58] <pcjc2> On SF, patches went to die
[01:58] <lifeless> thats awesome
[01:58] <pcjc2> We had to shut our -dev email list down to open membership due to Signal to noise ratio problems
[01:59] <pcjc2> LP bug comments have proved to be the -dev list we never had. Active, constructive discussion of the item in hand - far less OT
[01:59] <pcjc2> (And this is still only a couple of days into  our migration and triage effort!)
[02:00] <pcjc2> I know SF is not a great shining example to compare against, but I've had several positive comments from users who far prefer LP as well now they've tried it
[02:01] <lifeless> I wonder if we have a page for quotes ;)
[02:08] <thumper> lifeless: I've got a question if you've got a minute
[02:10] <pcjc2> Feel free to quote me if you  want
[02:10] <pcjc2> Really -> bed now, night!
[02:10] <LPCIBot> Project devel build (348): STILL FAILING in 3 hr 26 min: https://hudson.wedontsleep.org/job/devel/348/
[02:11] <lifeless> thumper: sure
[02:11] <thumper> lifeless: it is about the +build urls
[02:11] <thumper> lifeless: that bigjools replied to on launchpad-dev
[02:12] <thumper> lifeless: he mentions three points, but only one (IMO) is valid
[02:12] <thumper> lifeless: is it worth fighting or just add redirects?
[02:12] <lifeless> whats the thread subject?
[02:13] <thumper> Url change for binary package builds
[02:13] <LPCIBot> Project db-devel build (262): STILL FAILING in 3 hr 28 min: https://hudson.wedontsleep.org/job/db-devel/262/
[02:20] <thumper> lifeless: are you reading or busy?
[02:21] <lifeless> thumper: its dinner/drinks time, so multiplexing
[02:21] <thumper> :)
[02:25] <lifeless> which one is valid ?
[02:27] <thumper> that people may keep ids around
[02:28] <thumper> my change altered ALL binary builds
[02:28] <thumper> not just ppa ones
[02:28] <thumper> so that is one of the three out
[02:28] <thumper> the second point I disagree with
[02:28] <lifeless> the api stability argument may matter if the build objects were exported in '1.0
[02:28] <lifeless> '
[02:28] <thumper> I talked with flacoste about it and he doesn't think it is an issue
[02:29] <thumper> he seemed to think that the url wasn't a big issue...
[02:29] <lifeless> I think we need to decide if our apis are stable or not
[02:29] <lifeless> changing the url space for existing 1.0 exported things is a big deal IMO
[02:29] <thumper> lifeless: in order to get this fixed probably best to just add redirects yes?
[02:30] <thumper> this is an oops issue for us
[02:30] <lifeless> I think it probably is best overall, yes.
[02:30] <thumper> ok
[02:30] <lifeless> it is more work
[02:30]  * thumper gets on it
[02:30] <thumper> it is
[02:30] <thumper> the work is complete as is
[02:30] <thumper> so yes, more work
[02:30] <lifeless> OTOH the work makes the transition smoother during the deploy
[02:30]  * thumper shrugs
[02:30] <thumper> I'll get it done
[02:31] <lifeless> cool
[02:45] <lifeless> thumper: hey, do you think you could land mkanat's loggerhead version bump?
[02:45] <thumper> lifeless: where is it?
[02:46] <lifeless> thumper: https://code.launchpad.net/~mkanat/launchpad/lp-loggerhead-update/+merge/45787
[02:47] <thumper> lifeless: yep
[02:47] <lifeless> awesome, thanks!
[02:50] <thumper> gah
[02:50] <thumper> can't lp-land it
[02:50] <thumper> as I don't have that branch
[02:50] <thumper> isn't there a setting to say "land that one"
[02:51] <lifeless> if there isn't, there should be a bug asking for that
[02:51] <lifeless> I'd be happy for such a bug to be triaged 'high'
[02:54] <thumper> lifeless: which plugin?
[02:55] <lifeless> lp-submit I think
[02:55] <lifeless> perhaps pqm-submit
[02:58] <thumper> lifeless: it is in the pqm plugin
[03:00] <thumper> lifeless: is devel open?
[03:00] <thumper> lifeless: topic says no
[03:03] <lifeless> oh
[03:03] <lifeless> hmm
[03:04] <lifeless> we should be able to open again as soon as we're qa'd up past the db merge.
[03:04] <lifeless> I guess that isn't in the process docs yet or something, or perhaps we aren't qa'd to that point yet.
[03:38] <lifeless> spm: hi
[03:38] <spm> lifeless: ho!
[03:39] <lifeless> spm: did you see my analyze request ?
[03:40] <spm> oh no. sorry. doing now.
[03:42] <lifeless> spm: new post - http://pastebin.com/raw.php?i=5HBhT6R4
[03:43] <spm> ugh. 15+ seconds on getting the explain analyse
[03:43] <lifeless> yes, thats why we're asking for it ;)
[03:43] <lifeless> data isn't in staging yet
[03:45] <spm> ew. https://pastebin.canonical.com/41741/ 1 minute just for the analyse.
[03:45] <lifeless> spm: 2 plox, + the actual result
[03:46] <spm> yah. 2nd runing now.
[03:50] <lifeless> . wireless . at . hotels .. sucks
[03:50] <spm>  count
[03:50] <spm> -------
[03:50] <spm>      9
[03:50] <spm> (1 row)
[03:50] <spm> lifeless: ^^
[03:50] <lifeless> ???????
[03:51] <lifeless> what was the second analyze?
[03:54] <spm> I didn't paste that? oops. one sec.
[03:54] <lifeless> note that I can see
[03:54] <spm> https://pastebin.canonical.com/41742/
[03:54] <spm> no I didn't meant to add it to the results above
[03:54] <spm> didn't <comma> meant to...
[03:58] <lifeless> around for a little?
[04:02] <lifeless> spm: can you try
[04:02] <lifeless> http://pastebin.com/LcA17msn
[04:04] <spm> lifeless: https://pastebin.canonical.com/41743/ I like that version better
[04:07] <lifeless> thanks spm
[04:07] <lifeless> me too
[04:29] <wgrant> Wow, KDE is stupid.
[04:29] <lifeless> ...
[04:29] <wgrant> I installed python-kde4 last night to try to debug the SIGCHLD thing.
[04:29] <wgrant> I just logged out and back in, and now the test suite causes a KWallet prompt followed by a segfault.
[04:29] <lifeless> \o/
[04:31] <StevenK> Clearly, KDE is for people who want a broken Launchpad.
[04:31] <wgrant> It's probably not Launchpad's fault that there's a segfault.
[04:31] <wgrant> That is entirely down to KDE :)
[04:35] <wgrant> spm: Hi.
[04:37] <spm> wgrant: yo!
[04:38] <lifeless> yoyo
[04:38] <wgrant> spm: Could you please run http://paste.ubuntu.com/552687/ on prod?
[04:40] <spm>  architecturetag | count
[04:40] <spm> -----------------+-------
[04:40] <spm>  ia64            |  1982
[04:40] <spm>  sparc           |  1983
[04:40] <spm> (2 rows)
[04:40] <spm> wgrant: ^^
[04:40] <wgrant> Damn.
[04:40] <wgrant> OK, thanks :(
[04:41] <StevenK> wgrant: pitti is having lots of fun with your VGA cable
[04:42] <wgrant> StevenK: Oh?
[04:42] <wgrant> spm: http://paste.ubuntu.com/552689/, if you could.
[04:43] <StevenK> wgrant: Martin's laptop with Natty, 52" TV in the hotel room, do you need a calculator?
[04:43] <wgrant> StevenK: Ah, of course, making Unity crash even more spectacularly than usual!
[04:45] <StevenK> wgrant: Sadly, it didn't crash once.
[04:45] <spm> wgrant: https://pastebin.canonical.com/41744/
[04:45] <wgrant> StevenK: It doesn't do anything *but* crash on my desktop.
[04:46] <wgrant> spm: aaaaaaaaaaaaaaaaa
[04:46] <spm> heh
[04:47] <StevenK> wgrant: Can has bug with a backtrace?
[04:47] <lifeless> wgrant: is bug 78845 still true?
[04:47] <_mup_> Bug #78845: publication process should allow per-release holds <lp-soyuz> <Launchpad itself:Triaged> < https://launchpad.net/bugs/78845 >
[04:48] <wgrant> StevenK: Possibly.
[04:48] <wgrant> StevenK: It's a bit hard.
[04:48] <wgrant> Because fglrx is awful.
[04:48] <wgrant> And likes to hang once Unity crashes.
[04:48] <StevenK> Oh, ye gods.
[04:48] <lifeless> well there is your bug
[04:48] <wgrant> Although even vanilla Compiz crashes sometimes now.
[04:48] <wgrant> It was stable in maverick :(
[04:48] <wgrant> lifeless: I don't think that's ever been true.
[04:49] <wgrant> And it's even less true now that security uploads don't go through the queue.
[04:50] <StevenK> I suspect you're on your own, masochist
[04:52] <wgrant> spm: https://pastebin.canonical.com/41745 :(
[04:52] <spm> wgrant: https://pastebin.canonical.com/41746/
[04:53]  * spm is an SQL bot
[04:53] <StevenK> Today, you are.
[04:53] <wgrant> spm: Thanks.
[04:53]  * wgrant goes off to cry in the Soyuz corner.
[04:53]  * StevenK cries at test_hasbuildrecords.py
[04:54] <StevenK> Yes, let's use broken sampledata that's been changed by use of removeSecurityProxy() and THEN push it through code. Ja. That would be fun.
[04:57] <wgrant> Oh, they're not actually in the primary archive.
[04:57] <wgrant> Phew.
[05:01]  * StevenK grumbles at sourcepackage.py
[05:01] <StevenK> # End of duplication (see XXX cprov 2006-09-25 above).
[05:01] <StevenK> What XXX by cprov?!
[05:02] <wgrant> StevenK: IIRC the original XXX is in one of the duplicates.
[05:02] <wgrant> But not the other.
[05:02] <StevenK> Must. Resist. Throwing. Laptop.
[05:03] <wgrant> It's a ThinkPad, it'll survive.
[05:03] <StevenK> If it's closed.
[05:03] <wgrant> True.
[05:06] <spm> so close it, throw, then re-open?
[05:07] <StevenK> Hah
[05:07]  * StevenK rings Chubb first ...
[05:27] <StevenK> (Pdb) print self.builds[0].build_farm_job.date_finished
[05:27] <StevenK> None
[05:27] <StevenK> Ah ha. Hello, smoking gun.
[05:29]  * wgrant pushes delayed copies off a cliff.
[05:36] <wgrant> spm: http://paste.ubuntu.com/552703/ <- last one, but there'll be about 4000 rows returned.
[05:36] <spm> do you need all 4k?
[05:37] <wgrant> Ideally.
[05:37] <wgrant> We have at least three variations of the bad data... I'm not sure how many more there are.
[05:39] <StevenK> spm: Only give him 3,999. He'll never know ...
[05:39] <spm> StevenK: you are kidding right? this is *wgrant* we're talking about.
[05:40] <StevenK> Bwahaha
[05:40] <spm> amusingly, it's 3969 rows
[05:40] <spm> well, lines of output.
[05:41] <wgrant> I was expected 3965 rows of data.
[05:41] <wgrant> Which is one off.
[05:41] <wgrant> Odd.
[05:43] <spm> wgrant: https://devpad.canonical.com/~spm/wgrant.out
[05:43] <spm> (3965 rows)
[05:44] <wgrant> spm: Thanks.
[06:09] <wgrant> Sounds like it's a good time to visit Brisbane.
[06:09] <wgrant> brb
[06:36] <wgrant> Is there a reason we don't have a PropertyCache-based memoization decorator yet?
[08:44] <adeuring> good morning
[09:00] <al-maisan> Good morning adeuring !
[09:00] <adeuring> Hi al-maisan!
[09:01] <al-maisan> how are things!
[09:01] <adeuring> sad to see you leaving
[09:01] <al-maisan> s/!/?/
[09:01] <al-maisan> adeuring: das ist der Lauf der Dinge :)
[09:02] <adeuring> althings are fine here. Looking forward to the epic
[09:03] <al-maisan> nice :) when and where is the epic taking place?
[09:11] <bigjools> morning
[14:13] <jcsackett> leonardr: what's the qa status of bug 686690? is it qa-able?
[14:13] <_mup_> Bug #686690: 1.8.0 breaks login_with() API compat with existing credentials files, and forces keyrings <desktop-integration> <qa-needstesting> <regression> <launchpadlib :Fix Committed by leonardr> < https://launchpad.net/bugs/686690 >
[14:14] <leonardr> jcsackett: yeah, it's qaed, let me just make sure the new version made it into launchpad
[14:14] <jcsackett> leonardr: ok. thanks.
[14:16] <leonardr> jcsackett: yeah, looks good
[14:17] <jcsackett> leonardr: excellent. that's the last bug blocking the deploy-stable queue.
[14:17] <jcsackett> thanks, leonardr.
[14:18] <leonardr> np
[15:00] <flacoste> jcsackett: do we have revision number?
[15:00] <jcsackett> flacoste: unless someone contacts me with something that needs RC and landing by midday today, r12177.
[15:00] <flacoste> jcsackett: how about doing a nodowntime today of 12176?
[15:01] <jcsackett> flacoste: i'm comfortable with that.
[15:01] <flacoste> jcsackett: can you make it happen?
[15:02] <mwhudson> we can't do nodowntime to codehosting yet can we?
[15:02] <jcsackett> mwhudson: i don't know. flacoste? ^^
[15:03] <jcsackett> i thought we could nodowntime anything that didn't require db changes, but i could be quite wrong.
[15:03] <flacoste> mwhudson: we can't no
[15:04] <mwhudson> ah well
[15:04] <flacoste> mwhudson: codehosting isn't part of nodowntime yet, we have a RT in progress to fix that though
[15:04] <mwhudson> cool
[15:04] <flacoste> jcsackett: soyuz FTP services, librarian and codehosting are not part of the nodowntime set
[15:04] <flacoste> because deploying there actually causes downtime to the services
[15:04] <bigjools> I had an idea about ftp services
[15:04] <flacoste> we have RTs to fix codehosting and librarian
[15:05] <bigjools> and ftp :)
[15:05] <flacoste> soyuz FTP services will require some engineering
[15:05] <flacoste> bigjools: i thought ftp was more than a RT?
[15:05] <bigjools> flacoste: given that we never, ever change the ftp code, I filed an RT to have 2 deployment trees, one for FTP and one for everything else
[15:05] <jcsackett> flacoste: let me scan the deployment queue again then and make sure we don't have anything that couldn't be nodowntime then--i haven't been reviewing the queue with that in mind.
[15:06] <bigjools> then we can roll out to germanium's "everything else" tree as part of the nodowntime set
[15:06] <lifeless> jcsackett: most things
[15:06] <flacoste> bigjools: that would be a nice incremental improvement!
[15:06] <lifeless> bigjools: we can do an active-passive mode for ftp / sftp
[15:07] <bigjools> flacoste: rt 43238 if you want to poke
[15:07] <lifeless> bigjools: wgrant seems to think its harder than that though, and was going to speak to you about it
[15:07] <bigjools> lifeless: I think it will be hard too
[15:07] <bigjools> since we can't interrupt in-progress uploads
[15:07] <lifeless> sure
[15:07] <lifeless> same constraint we have on appservers
[15:08] <bigjools> so needs a bit of engineering - my 2-tree solution is much nicer I think
[15:08] <lifeless> I think its a good short term fix, I don't think its a good long term fix because we want to eliminate skew
[15:08] <bigjools> the only problem after that is worrying about a SPOF
[15:08] <bigjools> skew?
[15:08] <jcsackett> lifeless: most things?
[15:08] <bigjools> there is no skew
[15:08] <bigjools> the ftp code never ever changes
[15:10] <lifeless> the losas monitor deployed versions
[15:10] <lifeless> if we have two deploys on a machine
[15:10] <lifeless> they will get warnings if one is not running the current deployed version
[15:10] <sinzui> I didn't see an announcement of a rollout last week or this week, but I see we did one
[15:10] <bigjools> lifeless: Tom was happy to do this when I asked him
[15:10] <bigjools> I am sure they can cope
[15:10] <lifeless> bigjools: I'm happy for it too, short term.
[15:10] <lifeless> I don't think its a sensible long term arrangement
[15:10] <mthaddon> lifeless: not exactly - that holds true for groups of servers (lpnet servers, for instance) but not across all services
[15:10] <bigjools> why?
[15:11] <bigjools> I'm not wedded to the idea if there's something better, but you need to explain :)
[15:12] <lifeless> because it adds a nonobvious surprise waiting to be a pain in the future - especially if we very rarely change it
[15:12] <lifeless> (its the rarely changed things which hurt most, because the knowledge gets lost)
[15:12] <mthaddon> bigjools: I'm assuming the long term idea is make sure those services are deployable in a no-downtime-load-balanced way so we can deploy to them without affecting customers
[15:12] <bigjools> can you expand on what the nonobvious surprise is?
[15:12] <mthaddon> the amount of code that changes during a monthly rollout
[15:12] <bigjools> mthaddon: yes, but that will need extensive code changes to make it work
[15:12] <lifeless> the nonobvious surprise would be having a lp tree that isn't deployed to
[15:13] <bigjools> lifeless: not true, it would be deployed once a month
[15:13] <mthaddon> bigjools: actually, I think it needs as much infrastructure changes (haproxy, etc.)
[15:13] <bigjools> mthaddon: not quite, see above about not interrupting in-progress sessions
[15:13] <lifeless> bigjools: thats a constraint we have on the web appservers too
[15:14] <mthaddon> bigjools: we'd have a way of monitoring haproxy to ensure we only stop services when there are no active sessions (as we are planning to do for the appservers)
[15:14] <bigjools> yes, so we'd need to change some code, like I said :)
[15:14] <lifeless> thats fine
[15:14] <lifeless> short term vs long term
[15:14] <mthaddon> bigjools: and we'd have a mechanism to be able to say "pretend you're down to haproxy so it doesn't send new connections to you, but don't actually stop doing anything yet"
[15:14] <bigjools> mthaddon: ah, if you can do it all in the haproxy then that's a different matter
[15:15] <lifeless> bigjools: the ftp thing is zope's ftp service right ?
[15:15] <bigjools> yes, but I want it to die horribly
[15:15] <bigjools> and replace it with twisted
[15:15] <lifeless> bigjools: all it needs is a <small> http site that starts giving 500's after SIGUSR2 is received
[15:15] <mthaddon> bigjools: yeah, about 80% of it is done in haproxy - just need a mechanism to be able to fool haproxy into thinking it's down when it's not (but when you want it to be shortly)
[15:15] <bigjools> we have sftp in twisted, adding ftp will be easy
[15:15] <lifeless> sure
[15:15] <lifeless> either way
[15:15] <bigjools> then the zope shit can die
[15:15] <lifeless> my point is - doing what you've proposed is a great short term hack
[15:16] <lifeless> but longer term - the goal is one rev of lp everywhere, on every deploy
[15:16] <bigjools> if that's the goal, cool
[15:17] <bigjools> bear in mind though that you'll struggle to do any of this on cocoplum
[15:17] <lifeless> and yeah we have to write code and elinminate SPOF's and so forth
[15:17] <lifeless> one step at a time :)
[15:17] <bigjools> since we can't interrupt the current publishing cycle easily
[15:17] <bigjools> well, we can, but it's nasty
[15:18] <bigjools> we need cocoplum and germanium *2 to eliminate SPOFs :)
[15:18] <lifeless> or to put their storage on the SAN
[15:18] <bigjools> that's part of it
[15:18] <lifeless> then we can have a cluster of boxes to do the processing logic
[15:19]  * lifeless handwaves furiously
[15:19] <bigjools> ha
[15:19] <lifeless> seriously: I know this is a fiendish scale problem
[15:19] <bigjools> we'd need to re-engineer a load of soyuz stuff to make in multi-processable
[15:19] <bigjools> s/in/it/
[15:19] <lifeless> and there are model issues where we can wedge stuff if something is uploaded twice
[15:19] <lifeless> and so forth
[15:19] <bigjools> yeah, not impossible, just hard
[15:19] <lifeless> I'm advocating a calm one step at a time, interleaved with feature work and bug fixes
[15:20] <lifeless> the benefits of getting this all done are substantial
[15:20] <bigjools> I certainly won't argue with that
[15:20] <lifeless> at the moment we're in a sort of phase 1: get to the point where we can upgrade all our code without user visible impact
[15:21] <lifeless> that gets rid of a bunch of software level spofs
[15:21] <lifeless> doesn't do scaling
[15:21] <lifeless> doesn't do hardware spofs
[15:21] <lifeless> doesn't do datacentre-redundancy etc
[15:21] <bigjools> I'm very concerned about the scaling when we need to host derived distros
[15:22] <lifeless> I'd be delighted to talk about current bottlenecks and see if we can come up with a minimal way to add pluggable capacity
[15:22] <bigjools> we can chop probably 90% of the database load the b-m generates if we use MQs
[15:24] <bigjools> I'll run you through it next week if you like
[15:24] <lifeless> cool
[15:25] <sinzui> Googles index is mutating. My search for "prescription" in Launchpad yielded 105 matches. I expected 8. I saw only 40. A minute later I searched an there was only 4 matches
[15:33] <jml> lifeless: yo
[15:33] <jml> lifeless: I wish to speak with you.
[15:54] <danilos> anyone has any suggestion on how to debug a "ShortListTooBigError: Hard limit of 1000 exceeded." API-prefetching problem on a non-API using page? i.e. how would I figure out what attribute is getting serialized?
[15:54] <danilos> oh, how stupid of me, SQL log tells me all I need to know
[16:04] <lifeless> jcsackett: looks like we can unfreeze trunk
[16:04] <lifeless> jcsackett: given that we're qa'd up through 12177
[16:05] <jcsackett> lifeless: yes. on the list of todos i'm working down right now.
[16:30] <sinzui> danilos: jcsackett, you coordinate your work. you may be doing duplicate work or bugs
[16:30] <sinzui> only one of you should use do_not_snapshot on the specifications attr for product and distribution
[16:32] <jcsackett> danilos: what bug are you looking at? i'm looking at bug 696913
[16:33] <_mup_> Bug #696913: Product:+edit or +configure-* error 'Cannot update project details: shortlist exceeded 1000' <oops> <projects> <Launchpad itself:In Progress by jcsackett> < https://launchpad.net/bugs/696913 >
[16:33] <danilos> jcsackett, oh, I just filed a new one because I've seen it in translations
[16:34] <danilos> jcsackett, I am only fixing valid_specifications, I haven't hit the milestones yet, but I am sure I would when I tried this fix out
[16:34] <sinzui> danilos: and lifeless filed one for blueprints. I think there may only be one bug.
[16:34] <danilos> sinzui, right, it looks like it is, it's a simple doNotSnapshot missing, or a more complex "do-not-snapshot-by-default" bug
[16:35] <jcsackett> sinzui, danilos: it's the IHasSpecificationsMixin; none of its collections are doNotSnapshot-ed, and it's included in Product, so anything with products can hit this.
[16:35] <danilos> sinzui, jcsackett: I am marking bug 701529 as duplicate of that
[16:35] <_mup_> Bug #701529: OOPS on Distribution:+configure-translations <oops> <Launchpad itself:In Progress by danilo> < https://launchpad.net/bugs/701529 >
[16:36] <danilos> jcsackett, I have a branch which fixes it for valid_specifications (including a test that lives in registry, though ideally we'd have a IHasSpecifications test)
[16:36] <sinzui> jcsackett: sprint maybe
[16:36] <danilos> jcsackett, https://code.launchpad.net/~danilo/launchpad/bug-701529
[16:36] <jcsackett> sinzui: you mean leave this to the sprint to discuss solution?
[16:37] <sinzui> jcsackett: also projectgroup. We can test that by trying to edit the description of launchpad-project
[16:37] <jcsackett> sinzui: oh, sprint, the model.
[16:37]  * jcsackett is all out of whack today.
[16:37] <sinzui> jcsackett: I think the problem interface is mixed into ISprint
[16:37] <jcsackett> sinzui: yeah. IHasSpecifications is used in several places.
[16:37] <danilos> jcsackett, sinzui: since I am in Dallas, I would be very happy to hand this over to you guys, but the branch I have is already landable imo, so if you want I can land at least that bit
[16:38] <sinzui> danilos: thanks.
[16:38] <jcsackett> danilos: sure, get it landed and i'll deal with any conflict in my branch when it hits.
[16:42] <danilos> jcsackett, I guess it's not going to fix the problem even for translations, so I might just leave it to you :)
[16:42] <jcsackett> danilos: sounds fine. i think i'll have mine ready to land by my EOD.
[16:42] <jcsackett> i was trying something fancy earlier, and it goofed, so i'm going for the less fancy more functional fix.
[16:51] <bac> hi benji
[16:51] <benji> hey bac
[16:51] <bac> benji: i created an egg for lplib 1.9.3 but am now getting other errors using ec2
[16:51] <bac> benji: http://pastebin.ubuntu.com/552890/
[16:53] <benji> bac: hmm, that surprises me; looking into it more
[16:54] <benji> bac: I bet you're running this over an SSH session.
[16:54] <bac> benji: i am.  as i always do.  :)
[16:55] <bac> benji: i can try on the machine to see if i can get past the auth step
[16:55] <StevenK> bac: If it isn't obvious, I'm in Dallas, so please ping me for the US reviewers meeting this week, and not the er, other one.
[16:56] <benji> bac: I bet it'll work.  I'm looking for a better fix while you try that.
[16:58] <bac> benji: it did
[17:08] <danilos> jcsackett, so, the branch I have does fix the problem for IDistribution:+configure-translations page (tried it out by cowboying to staging)
[17:08] <jcsackett> danilos: cool. if you want to land it, i don't think we'll have a merge conflict down the road. my branch tackles the same bit, plus some other collection fields and milestone stuff.
[17:10] <benji> bac: you should be able to get GNOME keyring to work over ssh by running export `dbus-launch`
[17:13] <danilos> jcsackett, right, I don't want to land it if you are creating a nicer lib/lp/blueprints test, so I'd let you be the judge: https://code.launchpad.net/~danilo/launchpad/bug-701529/+merge/45886
[17:14] <danilos> jcsackett, if you by any accident feel an urge to approve that, I'll land it, but if you've got a better solution (I don't doubt you do :), I'll just scrap it
[17:15] <jcsackett> danilos: i am adding blueprints tests, but i don't think they prohibit your landing that--two extra fields on that test aren't a testing performance issue, imo.
[17:16] <jcsackett> danilos: if you would rather abandon it though and enjoy Dallas, i don't think you have to land it.
[17:16] <danilos> jcsackett, it's more of a "localization issue": you break non-blueprint tests by changing blueprints, which is something I hate
[17:16] <danilos> jcsackett, so, I'll scrap it (it'd still be hard to enjoy Dallas :)
[17:16] <jcsackett> danilos: heheh. dig.
[17:34] <timrc> bigjools: Is there a proposed Round 2 of testing for https://dev.launchpad.net/LEP/DerivativeDistributions -- I'd be interested in participating
[17:36] <bigjools> timrc: nothing planned, but if you have any comments I'll gladly take them
[17:40] <timrc> bigjools: is there something demoable regarding this feature?
[17:40] <bigjools> timrc: no, we're a long way off
[17:40] <timrc> bigjools: I'm generally interested as an OEM stakeholder
[17:41] <bac> benji-lunch: that worked, thanks
[18:03] <jml> sinzui: ping
[18:03] <sinzui> hi jml
[18:03] <jml> sinzui: hi
[18:04] <jml> sinzui: I'm just looking at https://lpstats.canonical.com/graphs/People/
[18:04] <jml> sinzui: it looks broken.
[18:05] <sinzui> Look like someone switching Lp to use Ubuntu's SSO
[18:05] <jml> sinzui: ahh, ok.
[18:05] <jml> sinzui: so I should pretty much ignore the "invalid" bit?
[18:05] <jml> sinzui: I'm trying to get a sense of number of users.
[18:05] <sinzui> I do not know really.
[18:06] <sinzui> but I think we need to broaded the range to ensure shipit nonsense is not mixed in too
[18:06] <sinzui> okay shipit is not a factor
[18:07] <sinzui> I wonder if this is looking a password. Lp does not use them, so I expect them to be null
[18:07] <jml> sinzui: also, do you know how https://lpstats.canonical.com/graphs/ProductsFlaggedOfficial/ is calculated? is it up to date wrt your recent changes?
[18:08] <sinzui> Yes, it is correct
[18:08] <sinzui> note the dips as we transitioned to the new rules
[18:09] <sinzui> jml: I looked at the people when I saw it hit a million. I think the number is legitimate!
[18:09] <jml> sinzui: that's pretty exciting :)
[18:11] <sinzui> I had a small anxiety attack at the time. Is there really that many in the communities, or do we have users who registered to do one single task
[18:11] <sinzui> jml: have you ever looked at how many users get karma every week
[18:11] <jml> sinzui: requently
[18:11] <jml> frequently, rather.
[18:11] <jml> sinzui: https://lpstats.canonical.com/graphs/KarmaBreakdownGlobalWeek/
[18:11] <sinzui> I have this nagging feeling we have a large silent failure in Lp
[18:12] <jml> sinzui: https://lpstats.canonical.com/graphs/KarmaBreakdownGlobalWeek/20070701/20110112/
[18:12] <jml> sinzui: how do you mean?
[18:12] <sinzui> less than 1% of registered users are active
[18:13] <jml> sinzui: yeah.
[18:13] <jml> sinzui: I think the number goes up a fair bit if you consider activity over a span longer than a week
[18:13] <sinzui> that is a bogus number because some people work on monthly and yearly scales
[18:14] <jml> sinzui: *nod*. it's about 20k per month
[18:14] <sinzui> sorry, message interpolated badly. My 1% remark is bogus
[18:14] <jml> sinzui: tell me more about your hypothesized large silent failure
[18:15] <sinzui> jml I can never find the graph branch. I am suspect we need to remove password from or account from the report to fix the numbers
[18:16] <jml> sinzui: bzr+ssh://bazaar.launchpad.net/~canonical-losas/tuolumne-lp-configs/trunk/
[18:16] <sinzui> thanks
[18:16] <jml> sinzui: it would also be nice to graph the number of projects that use *any* component officially
[18:16] <sinzui> jml: I suspect many users try Lp, then silently leave. Some maybe to report a bug, some to report a project.
[18:17] <jml> sinzui: yeah. I believe that.
[18:18] <lifeless> oh yeah
[18:18] <lifeless> did you guys see pcjc2's comments about launchpad last night?
[18:18] <mwhudson> about how it had suddenly made his project awesome?
[18:19] <sinzui> jml: I am talking with mrevell and gary next week about reintroducing the Lp setup profile workflow. We lost it going to Ubuntu SSO and I think it needs to be re-imagined to introduce Lp to users
[18:19] <mwhudson> or something
[18:19] <lifeless> mwhudson: yes
[18:19] <jml> lifeless: I didn't.
[18:19] <lifeless> 14:57 < pcjc2> Btw.. in case you didn't hear it enough - Launchpad is AWESOME
[18:20] <lifeless> 14:57 < lifeless> thanks :)
[18:20] <lifeless> 14:58 < pcjc2> Moving from Sourceforge to LP with our bugs has TOTALLY reinvigorated our two projects
[18:20] <jml> rockin'
[18:20] <lifeless> 14:58 < pcjc2> Contributors are coming out of the woodwork with patches, participating in review, reworking their patches
[18:20] <lifeless> 14:58 < pcjc2> On SF, patches went to die
[18:20] <lifeless> 14:58 < lifeless> thats awesome
[18:20] <lifeless> 14:58 < pcjc2> We had to shut our -dev email list down to open membership due to Signal to noise ratio problems
[18:20] <lifeless> 14:59 < pcjc2> LP bug comments have proved to be the -dev list we never had. Active, constructive discussion of the item in hand - far less OT
[18:20] <lifeless> 14:59 < pcjc2> (And this is still only a couple of days into  our migration and triage effort!)
[18:20] <poolie> excellent
[18:20] <lifeless> 15:00 < pcjc2> I know SF is not a great shining example to compare against, but I've had several positive comments from users who far prefer LP as well now they've tried it
[18:20] <lifeless> 15:01 < lifeless> I wonder if we have a page for quotes ;)
[18:20] <poolie> you should post it to the blog
[18:20] <lifeless> 15:08 < thumper> lifeless: I've got a question if you've got a minute
[18:20] <lifeless> 15:10 < pcjc2> Feel free to quote me if you  want
[18:20] <jml> that's awesome
[18:20] <poolie> tagged 'feedback'
[18:20] <jml> I will quotethat
[18:20] <jml> lifeless: and yes we do have a page for that sort of thing.
[18:21] <poolie> http://www.google.com.au/search?q=launchpad+testimonials none are us
[18:22] <jml> yes.
[18:22] <jml> I'll get my SEO team on to that
[18:22] <StevenK> Should we put that on help.lp.net ?
[18:22] <StevenK> Since, er, dev doesn't seem the right place
[18:22] <poolie> mm, i really think the blog is better
[18:23] <poolie> it's not user help information
[18:23] <poolie> or only for developers
[18:23] <poolie> or perhaps somewhere like +tour, but less dusty
[18:23] <StevenK> TBH, I personally feel blogs aren't static, but ...
[18:23] <StevenK> The bikeshed should be purple, clearly. :-P
[18:23] <poolie> "on this day, pcjc2 said..."
[18:23] <poolie> it's a moment in time
[18:23] <poolie> you can tag them to find what was said
[18:24] <jml> internally, https://wiki.canonical.com/Launchpad/Testimonials
[18:24] <lifeless> 'in the past, our users have said...'
[18:24] <poolie> but yeah, it's a bit bikesheddy
[18:24] <lifeless> it's a story about how we are perceived :P
[18:24] <StevenK> I don't think testimonials are a moment in time ... they should be current
[18:24]  * lifeless doesn't care
[18:24] <sinzui> lifeless: about Lp admins should not have nomination rights, is this because they own a team?
[18:25] <lifeless> sinzui: I was going off the data in the bug
[18:25] <lifeless> sinzui: owning a team would speak to the owner/operator split we were discussing the other month
[18:25] <poolie> oh, this is the person filing all these bugs recently
[18:25] <lifeless> sinzui: but I don't know if thats the case here
[18:26] <sinzui> lifeless: exactly my thinking. I have noted that it is challenging in Lp to learn what team I own since I am not always a member
[18:27] <poolie> really, wow
[18:28] <lifeless> sinzui: you might like to note that this may be 'they own' rather than 'they are admins' but that we need more data to diagnose.
[18:28] <lifeless> sinzui: or I can if you like, but I have about 60 bug tabs open and that one closed already
[18:30] <lifeless> StevenK: yo
[18:30] <lifeless> bug 261971
[18:30] <_mup_> Bug #261971: Add debconf preseed for architecture tag <Launchpad Auto Build System:Triaged> < https://launchpad.net/bugs/261971 >
[18:30] <sinzui> lifeless: I will look into it.
[18:30] <lifeless> sweet, thanks
[18:32] <StevenK> lifeless: Uh?
[18:32] <lifeless> StevenK: is that still needed? crack? done?
[18:32] <StevenK> It's crack cut with arsenic, and lpia is dead, please kill it.
[18:33] <StevenK> lamont may disagree
[18:33] <StevenK> lifeless: I've pinged lamont, lesse ...
[18:34] <lifeless> so have I
[18:35] <StevenK> Haha. We pinged in different channels, for hilarity
[18:36] <lifeless> where did you ping?
[18:44] <lifeless> StevenK: surely bug 277550 is a dupe of some master 'does not support alternate dependencies' thing ?
[18:44] <_mup_> Bug #277550: binary package status page does not display alternate dependencies <lp-soyuz> <Launchpad itself:Triaged> < https://launchpad.net/bugs/277550 >
[18:45] <StevenK> We do support alternative dependencis
[18:45] <StevenK> It's a bug in the model
[18:46] <lifeless> I don't follow
[18:46] <lifeless> do we have duplicate code?
[18:46] <StevenK> lifeless: I'm trying to fix something, so I'm trying hard to not context switch
[18:46] <lifeless> ok
[18:47] <StevenK> lifeless: wgrant would be the best person to ask about that bug
[18:47] <lifeless> thnaks
[18:59] <jml> lifeless: a while ago you asked me for a public graph of the number of bugs tagged oops & timeout
[18:59] <jml> lifeless: I haven't done that yet.
[18:59] <jml> lifeless: do you still want it?
[19:00] <lifeless> would be nice. regression too
[19:01] <jml> ok.
[19:01] <jml> I'll try, but I won't be hurt if someone else does it before I do.
[19:01] <timrc> offhand, launchpad supports arbitrary length tags?
[19:01] <timrc> the limit would just be a hard limit (e.g. db space)
[19:02] <jml> flacoste: hi
[19:06] <jml> flacoste: actually, off to lunch, emailed instead.
[19:54] <lifeless> mwhudson: bug 613806
[19:54] <_mup_> Bug #613806: SSH server should forward OOPS code to bzr client <codehosting-ssh> <lp-code> <Launchpad itself:Triaged> < https://launchpad.net/bugs/613806 >
[19:54] <lifeless> mwhudson: IIRC another identical to that was filed recently;
[19:55] <lifeless> is that right?
[20:02] <lifeless> sinzui: there are now no untriaged bugs
[20:02] <sinzui> \o/
[20:02] <lifeless> well
[20:02] <lifeless> none triaged + importance unknown
[20:03] <lifeless> you may find it entertaining that some bloke 'Curtis' made a lot of them triaged (from confirmed) without setting an importance
[20:03] <sinzui> I investigated one oh your bugs in November, I need to find the related bugs to explain what has to change to close the bug
[20:03] <sinzui> I suck
[20:03] <lifeless> :P
[20:03] <sinzui> ajax takes too long to wait for a response
[20:16] <lifeless> sinzui: whats the sso bug tracker ?
[20:17] <sinzui> lifeless: do you mean project? canonical-identity-provider
[20:17] <lifeless> thanks
[20:17] <lifeless> we still have 176 'new / incomplete' bugs
[20:18] <sinzui> We cannot set bug expiration until we fix the snapshot/shortlist bug
[20:18] <lifeless> I suspect many shouldn't expire
[20:18] <lifeless> kindof
[20:18] <lifeless> e.g. bug 484712
[20:18] <_mup_> Bug #484712: Potential synchronisation of comments between Launchpad bugs via remote bug trackers <bugwatch> <lp-bugs> <Launchpad itself:Incomplete> < https://launchpad.net/bugs/484712 >
[20:18] <lifeless> is just gmb being slack on answering :)
[20:20] <sinzui> oh good. I thought he just had a junk filter on my bug comment
[20:20] <lifeless> he may, but I'm sure its not personal :)
[20:21] <sinzui> I mark all bug mail from ~/sinzui as junk. It would not surprise me if everyone did
[20:23] <sinzui> lifeless: the next cleanup should be all non-released bugs targeted to a past release. It irks me to see a bug I want fixed that says it is in progress and targeted to a milestone from two years afo
[20:43] <lifeless> flacoste: is deryck around atm ?
[20:43] <flacoste> lifeless: he's out today
[20:43] <flacoste> lifeless: we'll be back with us tomorrow
[21:03] <lifeless> wgrant: hey
[21:29] <jml> is there an automated way of finding out, say, what the owner of a distribution can do?
[21:29] <lifeless> other than try it?
[21:29] <jml> that's not automatic.
[21:29] <mwhudson> jml: i think roughly speaking no
[21:30] <jml> I think I've figured out how I could write something that would be close.
[21:30] <mwhudson> you can look zcml that has launchpad.Edit on distribution contexts
[21:30] <mwhudson> i think that's fairly close
[21:33] <jml> I was thinking of going through security.py class by class
[21:33] <thumper> sinzui: ping?
[21:34] <sinzui> hi thumper
[21:34] <thumper> sinzui: I have a branch that needs a quick UI review
[21:34] <thumper> sinzui: I have a pic even
[21:35] <jml> filling out https://wiki.ubuntu.com/LaunchpadPermissions fwiw
[21:35] <thumper> sinzui: https://code.launchpad.net/~thumper/launchpad/show-recipe-upload-issue/+merge/45804
[21:35] <sinzui> I saw. I asked my two mentees to look
[21:35] <sinzui> salgado was not about. mrevell did not reply.
[21:36] <sinzui> thumper: r=me, I will update the MP
[21:36] <thumper> sinzui: ta
[21:39] <jml> of course, there's stacks of code that has 'check_permission(obj, "launchpad.Edit")' manually
[21:39] <jml> huwshimi: hi
[21:39] <huwshimi> jml: Hello
[21:39] <jml> huwshimi: how's things?
[21:40] <leonardr> james_w, we would like to take another shot at including a more recent version of launchpadlib in natty. do you have time to talk about this?
[21:40] <huwshimi> jml: Good thanks. I'm all set up and managed to work on a couple of bugs yesterday
[21:40] <james_w> leonardr, would Monday work?
[21:40] <jml> huwshimi: sweet.
[21:40] <jml> huwshimi: so you're able to run up a local instance and what not?
[21:40] <leonardr> james_w: i'll be at a sprint on monday, so it would have to be after hours
[21:40] <huwshimi> jml: Yeah, all good.
[21:41] <james_w> leonardr, as will I :-)
[21:41] <leonardr> james_w: we won't be at the same sprint, will we??
[21:41] <james_w> leonardr, I'm attending 2.5 days of the megathunderdome
[21:41] <leonardr> all right, we'll talk then
[21:53] <thumper> wgrant: you there?
[21:56] <thumper> I cleaned out my eggs directory to get buildout to regenerate them (and clean out old versions)
[21:56] <thumper> now the test runner complains about a lot of missing modules
[21:56] <thumper> ImportError: No module named conch.interfaces
[21:56] <thumper> ImportError: No module named conch.checkers
[21:57] <thumper> ImportError: cannot import name BadRequest (from lazr.restfulclient.errors)
[21:57] <thumper> ImportError: cannot import name ClientError (from lazr.restfulclient.errors)
[21:57] <thumper> hmm...
[21:58] <thumper> I've not merged devel
[21:58] <thumper> lets try that
[22:01] <lifeless> bigjools: hi
[22:01] <lifeless> bigjools: or is that a false image of you?
[22:01]  * bigjools is not really here
[22:01] <lifeless> bigjools: ah.
[22:02] <lifeless> bigjools: I was considering suggesting wgrant look at bug 641338 today
[22:02] <_mup_> Bug #641338: Archive:EntryResource:syncSource timeouts <lp-soyuz> <pg83> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/641338 >
[22:02] <bigjools> he's busy with other stuff
[22:02] <lifeless> I think it can be improved fairly easily
[22:02] <lifeless> ok
[22:02] <lifeless> thanks
[22:02] <bigjools> but when he's done those, sure
[22:02] <bigjools> it's the same underlying problem as the copy package page
[22:03] <bigjools> aka the copy checker
[22:08] <wgrant> lifeless: I've been sorting out the whole copier mess this week.
[22:08] <wgrant> lifeless: The code is pretty shit. And the tests are worse.
[22:08] <wgrant> I do have a slight performance improvement, though.
[22:10] <lifeless> \o/
[22:11] <wgrant> And cutting another few hundred queries of it is easy once I detangle a few things.
[22:11] <wgrant> But I keep running into more bugs :/
[22:11] <wgrant> And then finding more broken data caused by them :/
[22:11] <lifeless> \o/
[22:15] <bigjools> wgrant: at some point we need to refactor the whole copy checking thing with the upload processor
[22:16] <thumper> yay, merge devel, and rebuild fixed testing errors
[22:18] <wgrant> bigjools: Yes, I was whinging to bigjools about bits of that last night.
[22:18] <wgrant> Er.
[22:18] <wgrant> You are bigjools.
[22:18] <wgrant> I am asleep.
[22:19] <bigjools> do I need to throw coffee at you?
[22:19] <thumper> bigjools: hi
[22:19] <bigjools> thumper!
[22:19] <thumper> bigjools: FYI, went with +buildjob and redirects
[22:19] <thumper> bigjools: all done now
[22:19] <bigjools> thumper: fantastic.  sorry I had to piss on your bonfire and all that
[22:19] <StevenK> Haha
[22:19] <thumper> bigjools: ah well
[22:20] <thumper> you can buy me lotsa alcohol next week to make up for it :)
[22:20] <bigjools> thumper: I know what it's like to have a branch ready to land and then someone points out a problem :)
[22:20] <bigjools> thumper: I am always up for alcohol
[22:46] <sinzui> bigjools: bac and I where wondering if these people really should have primary archives: http://pastebin.ubuntu.com/552994/
[22:46] <bigjools> sinzui: !!!!
[22:46] <sinzui> ^ bigjools, these people are the missing people from team.participants. a logic bug excludes them because they have primary arhives
[22:46] <bigjools> how the ...
[22:47] <bigjools> I think we need to find out how they got like that
[22:47] <bigjools> sinzui: that's a serious bug
[22:47] <sinzui> look at the ids, these are super old
[22:48] <bigjools> maybe
[22:48] <sinzui> well the first half are and I was looking for them in a list of members
[22:48] <bigjools> sinzui: well indeed, the last 2 are recent
[22:49] <bigjools> I suspect the others are badly migrated data
[22:49] <sinzui> And they moonlight as Launchpad gods
[22:50] <bigjools> feel free to fix those
[22:50] <bigjools> wgrant can give you an eyeball
[22:53] <wgrant> It's not *that* insane.
[22:53] <wgrant> bigjools: Do you know how much a distro likes being without a primary archive?
[22:54] <wgrant> I don't think much will care.
[22:54] <wgrant> It's not at all surprising that they're there, though.
[22:54] <wgrant> Creating a distribution creates a primary archive.
[22:55] <sinzui> That explains only three people in the list
[22:55] <sinzui> select archive.id, archive.purpose, archive.name, person.name from archive join person on archive.owner = person.id where purpose = 1 and person.teamowner is null;
[22:55] <wgrant> sinzui: Why?
[22:55] <wgrant> sinzui: The owner can be customised at creation-time.
[22:55] <bigjools> hmmm had not thought about the distros
[22:56] <sinzui> Only kiko + losas could create distros
[22:56] <bigjools> easy to double check if they're attached ti one
[22:56] <wgrant> bigjools: They are.
[22:56] <sinzui> I do not think kiko can do it anymore
[22:56] <wgrant> I actually was looking at old archives last night, as it happens.
[22:56] <sinzui> Chex made a distro a few weeks ago. That accounts for his archive
[22:56] <bigjools> makes sense
[22:57] <wgrant> sinzui: The owner has to be specified when creating a distro.
[22:57] <wgrant> sinzui: So anyone can own the archive.
[22:57] <wgrant> And I recognise a few other names there.
[22:57] <wgrant> Like joejaxx.
[22:58] <sinzui> Even though the distro cannot use the archive, and most registered are not debian based
[22:58] <sinzui> we suck
[22:58] <wgrant> sinzui: Sure. We should nuke all primary archives except 1 and 3.
[22:58] <wgrant> Once I check that the code is OK with it.
[22:58] <wgrant> And we should also fix the code to stop creating them.
[22:58] <wgrant> Because that's pointless...
[22:59] <wgrant> sinzui: Oh, that is a nice bug.
[23:00] <wgrant> I guess it works properly if you move the PPA restriction into the subselect?
[23:00] <jml> oh hey
[23:00] <sinzui> I think so
[23:00] <jml> we were talking about distro roles today
[23:01] <jml> and I was reminded that bug nomination has a duplicated-but-different set of logic for checking if someone can upload a package.
[23:01] <wgrant> Yeah.
[23:01] <wgrant> I looked at refactoring that, but it is not as trivial as your method makes it look.
[23:02] <jml> I got a vague sense of difficult-ness when looking at it today
[23:02] <jml> wgrant: what are the issues?
[23:02] <wgrant> jml: It has been a year. I don't quite recall, sorry.
[23:02] <jml> fair enough :)
[23:02] <wgrant> Let me look again.
[23:03] <wgrant> But twice now I've gone to try to refactor it, then remembered the problem I encountered the previous time.
[23:04] <jml> heh
[23:05] <wgrant> It doesn't really jump out to me at the moment. But the BugNomination code will still have to work out the component and stuff itself.
[23:05] <wgrant> And it's not obvious what to do about pockets.
[23:08] <wgrant> sinzui: It looks like there are two places it might crash if we remove the primary archives: the queue view (which shouldn't be used anyway, and isn't linked), and the bug nomination stuff might as well.
[23:08]  * wgrant tests on DF.
[23:08] <sinzui> bugger
[23:09] <sinzui> bug nomination code is so damn crifty
[23:09] <sinzui> crufty
[23:09] <wgrant> You clearly haven't seen Soyuz.
[23:10] <sinzui> I have. But I was trying to write a bug nomination test last month and discovered that factory setups do not work because the code ignored the env. It uses launchbag
[23:11] <mwhudson> is there any good reason (other than amount of work) that the launchbag can't be killed?
[23:11] <wgrant> Yup.
[23:11] <jml> wgrant: I think it might be OK to change the behaviour of the bug nomination code
[23:11] <wgrant> mwhudson: It's pretty hard.
[23:11] <jml> not sure though
[23:11] <wgrant> mwhudson: Since vocabs use it.
[23:11] <jml> wgrant: but my suspicion is that even if it doesn't do the same check now, it ought to be.
[23:11] <mwhudson> wgrant: what for?
[23:12] <mwhudson> current user should be got from the interaction
[23:12] <wgrant> mwhudson: Bug nominations, for one thing!
[23:12] <mwhudson> wgrant: hee
[23:13] <wgrant> mwhudson: The nominatable series vocab checks launchbag for the distribution or product to look at.
[23:14] <sinzui> I am all for removing launchbag. I favour completing the apocalypse first
[23:14] <thumper> +1
[23:14] <wgrant> sinzui: But the apocalypse is being reversed...
[23:14] <sinzui> It took 4 years to remove general view
[23:14] <sinzui> wgrant how so
[23:15] <wgrant> sinzui: We split the codebase by app 18 months ago.
[23:15] <wgrant> But a month ago we merged the app-split bugs.
[23:15] <jml> different things
[23:15] <jml> it's still one code base
[23:15] <sinzui> wgrant: not the same
[23:15] <jml> the apocalypse is about making it easier to navigate in the best way we could think of at the time
[23:15] <jml> still makes sense to complete
[23:16] <sinzui> the issue is about defining contracts and responsibilities in the code
[23:17] <jml> btw
[23:17] <jml> I also had a bit of a chat w/ cjwatson today about NewReleaseCycleProcess
[23:18] <jml> remind me to talk about that next week
[23:18] <jml> also, some people here might be interested in https://wiki.ubuntu.com/LaunchpadPermissions
[23:20] <sinzui> jml: I sent a revised brain dump of my understanding to mrevell recently. I advised that each team lead look at the list because every time I think I have a definitive list, someone cites some obscure location in the code that provides another permission
[23:21] <jml> sinzui: that's the advantage of having a shared, editable document :)
[23:22] <sinzui> I also advised we ask reviewers to ask developers to update documentation when a permission is changed
[23:22] <jml> sinzui: that's pretty hard to do
[23:22] <sinzui> eg. bug nomination rules changed a few moths ago, and now we are getting Ubuntu questions about it
[23:24] <sinzui> The document cited in this question cannot be changed by me: https://answers.launchpad.net/launchpad/+question/140509
[23:26] <jml> sinzui: that's unfortunate, but it's no reason not to keep *a* document up to date
[23:27] <sinzui> I agree. Collective code in the case requires a collective document
[23:31] <jml> sinzui: also, the Ubuntu guys need to be able to figure out how to run their project. :)
[23:36]  * spm asks jml how much he's prepared to pay to not get widely quoted on that one ;-)
[23:39] <jml> spm: I will not be beholden to blackmailers.
[23:49] <LPCIBot> Yippie, build fixed!
[23:49] <LPCIBot> Project devel build (349): FIXED in 4 hr 48 min: https://hudson.wedontsleep.org/job/devel/349/