mwhudsonthe 503 page change was the one that uses flash for xdr?00:00
wgrantmwhudson: Yes.00:00
wgrantmwhudson: To get at identi.ca/twitter feeds.00:00
wgrantYes, the Web is broken... but Flash is not a solution :)00:01
lifelessmwhudson: wgrant: it uses flash as a fallback, not by default00:11
lifelessmodern browsers with javascript enabled will just work without the flash00:11
lifelesshowever failing tests is a good reason to rollback :)00:11
wgrantI wonder if those profiling changes weren't meant to land at all.00:17
wgrantThey seem somewhat insane and irrelevant.00:17
* wgrant rolls it all back.00:17
huwshimiwgrant: How did you figure out which js was using mochikit? Or did you just remember?00:19
wgranthuwshimi: I remembered seeing some bits of it around... grepping for it is going to be difficult :/00:20
lifelessexplain select * from oops_oops;00:23
lifeless                               QUERY PLAN00:23
lifeless Seq Scan on oops_oops  (cost=0.00..1922179.42 rows=27723142 width=464)00:23
lifeless27M rows.00:23
mwhudsonah, i was wondering why you needed to see the query plan for that statement :-)00:23
lifelessmwhudson: faster than count(*) :)00:24
mwhudsondifferent answer too, i guess?00:24
mwhudsonstatistics vs actual00:25
lifelessthe stats at last update know the total live row count00:25
lifelessvs instantaneous row count00:25
* mwhudson slams head into desk about interactions in lp's tests00:25
lifelessmwhudson: welcome !00:25
wgrantSensible interfaces? What are they?00:26
lifelessmwhudson: what got you onto this?00:26
mwhudsonhttps://bugs.launchpad.net/launchpad/+bug/631884 basically00:27
_mup_Bug #631884: multiple variables pointing to FeatureController get out of sync easily <feature-flags> <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/631884 >00:27
lifelessah yes00:27
lifelessthere is another one about get_request_timeline00:27
lifelesssame issue00:27
mwhudsonyeah, features are a bit simpler, more localized00:28
mwhudsonso attacking them first00:28
lifelessI like the sound of that00:28
mwhudsonand my change isn't directly related to the problem i was head desking about, or at least my change doesn't make any difference00:29
mwhudsonbut for example, it doesn't seem we make any systematic effort to ensure that an interaction doesn't outlive a testcase00:29
mwhudsonalso, TestCaseWithFactory sets up an interaction, TestCase doesn't00:30
mwhudsonwhich is a bit ... dunno, something00:30
lifelessdoesn't suprise me00:30
lifelessif I may make a suggestion00:30
lifelessa fixtures.Fixture to push and pop interactions would be very useful00:30
mwhudsonyes probably00:31
mwhudsoni think also a pithy email and wiki page along the lines of "10 things every launchpad developer absolutely positively has to know about interactions and participations or else" is forthcoming00:31
lifelesswgrant: mwhudson: can has review? ^01:02
mwhudsonoh right, profiling tests are/were broken on devel, right?01:20
mwhudsonand guess what!01:21
mwhudsonthread local variables are involved01:21
mwhudson    def assertCleanProfilerState(self, message='something did not clean up'):01:22
mwhudson        """Check whether profiler thread local is clean."""01:22
* mwhudson chucks his feature flag thing into ec201:25
lifelessok pgsql, thats just daft01:29
lifelessor rather, like queries really really suck01:30
lifelessexplain select pathname from oops_oops where pathname like '/srv/launchpad.net-logs/production/wampee/2010-05-24/%' order by pathname;01:31
lifeless                                              QUERY PLAN01:31
lifeless Sort  (cost=2000294.79..2000301.89 rows=2841 width=68)01:31
lifeless   Sort Key: pathname01:31
lifeless   ->  Seq Scan on oops_oops  (cost=0.00..2000131.82 rows=2841 width=68)01:31
lifeless         Filter: ((pathname)::text ~~ '/srv/launchpad.net-logs/production/wampee/2010-05-24/%'::text)01:31
lifelesspossibly too-few statistics for the table size01:31
wgrantIs there an index?01:31
lifeless    "oops_oops_pathname_key" UNIQUE, btree (pathname)01:31
lifelessof course, its 5.5GB due to bloat.01:32
lifelesswow, disk sorts. funky.01:35
lifeless explain select pathname from oops_oops where pathname like '/srv/launchpad.net-logs/production/wampee/2010-05-24/%' and date between '2010-05-24'::timestamp - '1 day'::interval and '2010-05-24'::timestamp + '1 day'::interval order by pathname;01:40
lifelessseems to do whats needed.01:40
lifelesswe may be better offer with an IN statement or a temp table.01:41
lifelessthough they can degrade to seq scans horribly quickly.01:41
lifelesstemp table is at leat reasonabl01:43
lifelessany reviewers in the house ? 8 liner ...01:44
mwhudsonlifeless: ah yes, my machine crashed after you pasted the link01:52
mwhudsonlifeless: link me up again?01:52
mwhudsoner, lunch time01:55
wgrantlifeless: So, do we have any plans to fix all these timeouts?02:23
lifelesswgrant: yes.02:23
wgrantlifeless: Any *specific* plans? :)02:42
lifelesswgrant: yes :)02:43
* StevenK waits for "* wgrant shakes lifeless until some plans fall out."02:44
wgrantPretty much!02:44
wgrant'cause we haven't made any progress in quite a while.02:45
wgrantApart from the batchnav thing.02:45
lifelessbug search is a prime candidate for solr02:45
wgrantAnd that's at least 18 months away.02:45
lifelessthats a different discussion02:45
wgrantIs it?02:46
lifelessbroad strokes: - tune implementations now (e.g. denorm, search schemas etc), and do a dedicated search engine if/when we reach the limits of that approach02:46
wgrantWe have some debilitating, embarrassing timeouts. We're making little to no progress on them.02:47
=== wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 253 - 0:[#######=]:256
lifelesswe have no engineers currently hacking on them02:48
lifelessthats the proximate cause of no progress02:48
lifelessfrancis is analysis *why* thats happening02:48
wgrantThat's been happening for months now.02:48
wgrantWe have not made critical bug progress for 4.5 months now.02:48
wgrantWe adopted the new critical bug process 8.5 months ago.02:49
wgrantMore than half the time the new process has been working, it has been failing,.02:49
lifelessyes, thats the evidence.02:50
lifelessnow, do you have an analysis of causes?02:50
wgrantWell, the evidence points to one.02:51
wgrantIt's not all of it.02:51
wgrantBut it is something.02:51
lifelesstoo many new criticals?02:51
wgrantThe maintenance squad configuration.02:52
lifelessplease expand on that :)02:52
wgrantIn particular, the week Teal stopped taking criticals, the downward trend stopped.02:52
wgrantWe stopped taking criticals just after the middle of May.02:52
lifelessthe problem started earlier02:53
lifelessso, that simply speaks to our ability to tread water02:53
wgrantSure, but we're no longer even close to treading water.02:53
lifelessthe key question is where are all the criticals coming from02:53
wgrantIt was clear we were in a terrible situation 3 months.02:54
wgrant3 months ago02:54
lifelessuntil we rule out maintenance efforts, we can't even thing about things like 'add more maintenance folk'02:54
wgrantHow long is this going to take?02:54
wgrantWe are getting into a steadily worse situation.02:54
wgrantAnd I've seen no acknowledgement that we actually have a crisis here.02:54
lifelessI don't think its a run around in circles crisis02:55
lifelessits a significant issue02:55
wgrantWhat is a run around in circles crisis, then?02:55
lifelessand francis is working on the analysis02:55
lifelesswgrant: one where stopping and thinking will not help the outcome02:55
wgrantThe new definition and process are clearly not working. Reasonable functioning of the new structure of the LP team depends entirely on the critical queue being exhausted.02:55
wgrantThe critical queue is being the opposite of exhausted.02:56
lifelesswgrant: yes, yes. Yes.02:56
wgrantTherefore the new structure of the LP team will not provide reasonable response and functionality.02:56
wgrantIt's been nearly a year.02:56
wgrantSomething needs to be done.02:56
lifelessso, lets detangle some things02:56
lifelessfirstly, the delta from old structure to new is *still*, even with this issue, positive: we -are- progressing both maintenance and development efforts with less scheduling headaches.02:57
wgrantHmm? For some definitions of maintenance, perhaps.02:57
lifelesswe haven't solved /all/ the issues we hoped to, but we have some.02:57
wgrantI would posit that maintenance is, on the whole, substantially worse than it was.02:57
wgrantUnless you restrict "maintenance" to "fixing timeouts and OOPSes"02:58
lifelesswgrant: and escalated bugs, which previously were a real headache, we are doing them more reliably and with less delay than previously.02:58
wgrantIn some cases, yes.02:58
lifelesswgrant: welcome to statistics.02:59
lifelessas for doing something.02:59
wgrantNow, I'm not saying that the new structure is bad. I'm saying that the current implementation is objectively not working.02:59
lifelessDo you propose we go off half cocked without analysing the situation ?02:59
wgrantNo, we need analysis.02:59
wgrantBut we needed analysis 3 months ago.02:59
wgrantThere is no analysis :(02:59
lifelessfrancis is doing one.02:59
lifelessWhats your beef with it ?02:59
lifelessI posit that you want it complete yesterday.03:01
wgrantIs it going to come to a conclusion?03:01
lifelessof course03:01
wgrantIt has been several weeks.03:01
lifelessof course03:01
lifelessits not a trivial undertaking, and the person doing it has other things (like .e.g qbr prep) that are higher priority03:01
lifelessit took 4 or so months to do the analysis around the restructure03:02
wgrantBut it's something that needs to be done quickly, so perhaps the terribly busy person isn't the ideal person to do it :)03:02
wgrantWe have a serious problem here, and the longer we take to analyse it, the worse it is getting.03:02
wgrantIt took months for people to admit that we had a problem.03:03
lifelessthe scale is changing slowly, the structure isn't.03:03
wgrantAnd now it is going to take months to analyse.03:03
lifelesswell, it may not take months, but its going to take more than a couple of days :)03:03
wgrantIt has already taken months.03:03
lifelessWhat is your objective in this discussion? To convince me its drop-dead urgent? It doesn't seem to be to me. Important - yes. Important enough that its being done right now? Yes. Run-with-knives urgent? no.03:04
lifelessthe situation we're in is broadly 'too much to do and not enough hands to do it with'; thats -normal-. Whats interesting here is that we have things we believe we -must- do, which we don't have enough hands to do.03:05
wgrantAnd the number of hands is decreasing :)03:06
lifelessAnd that looks abnormal when you consider the definition of the things we -must- do is pretty modest [plug regressions, plug scaling issues, do escalated tasks]03:06
lifelessthe stuff francis has noted so far is that its mainly self inflicted03:07
lifelessIIRC > 1/2 the bugs analysed so far have been caused by feature work03:07
lifelessbut he hasn't analysed enough to be confident that its indicative in the big picture03:08
lifeless*IF* it is, then the thing to point at is cost of change: particularly hidden costs.03:08
wgrantHm, is this analysis progress visible somewhere?03:08
lifelessAnd that will come back to code structure, which gets back to what reviews are [perhaps failing] to do. For instance.03:09
wgrantI'd not heard any of this.03:09
lifelessI don't know where his working notes are03:09
wgrantIt was not clear that any progress had been made whatsoerver.03:09
lifelessI talk with him weekly03:09
lifelessStevenK: ++03:10
lifelessnuking sampledata. Love it.03:10
StevenKI'd remove all of it if I think it had any hope in hell of passing ec2.03:11
lifelessoops-tools is truely painful to hack on03:12
lifelessslow tests + no way (that I can see) to run just one.03:12
lifelesswgrant: StevenK: I need a brief teddy bear about removing oops prefixes and the impact on fauly analysis03:20
lifelesscould either of you spare 10 minutes?03:20
wgrantlifeless: Sure.03:20
StevenKPublic holiday and about to brave the daystar03:20
lifelesswgrant: skype ?03:21
huwshimiwallyworld: Do you know much about the names of the permission levels on the +managedisclosure page (e.g. Observer, Restricted Observer)? Are they used elsewhere in Launchpad?03:56
wallyworldhuwshimi: no, this is new terminology afaik03:57
huwshimiwallyworld: OK, I'm just trying to figure out if there are any better labels we could come up with. Something a bit more descriptive. E.g. Restricted Observer can only view subscribed so they could be labelled "Subscriber"03:59
huwshimiwallyworld: Any suggestions?03:59
wallyworldhuwshimi: i think we are trying to get away from conflating visibility with subscriptions04:00
huwshimiwallyworld: OK, that makes sense. I think I mostly have reservations about the word "observer".04:01
wallyworldit's hard to come up with a one or two word label that conveys the intended meaning04:01
wgrantSame with "disclosure"04:01
wallyworldi think though that once the terminology becomes socialised, it will be understood04:02
wgrantWe're very good at using bad terminology.04:02
lifelessyou could just not label them.04:02
lifelessand instead describe their rights04:02
wallyworldperhaps we need good inline or popup help to explain what it means04:02
lifeless[full access]\n* person\n* person ...04:02
lifeless[access to listed objects only]\n* person\n* person04:03
wallyworldlabels are needed if we want to do a choice list selection widget though04:03
huwshimiwallyworld: Yeah I was working on that when I decided I really wanted to think about the labels04:03
huwshimiwallyworld: Yeah that's right04:03
lifelesswallyworld: do we need a label for 'class of role' really?04:03
huwshimiwallyworld: Or for things like "Add an Observer" links04:03
wgrant"Grant access"?04:03
lifelesswallyworld: or just a phrase that explains04:03
wallyworldi think we need 3 things: 1. a good label, 2. a short sentence that can go in brackets after, and 3. verbose popup help text04:04
wallyworldthere are 2 role classes that can see stuff because of access being explicitly granted, vs one access class that can implicitly see everything because of the project role04:06
wallyworldso that makes it a bit harder to come up with concise labels04:06
lifelessthere are?04:06
wgrant"Project role => access" is a myth that I hope to dispell soon.04:07
lifelesssorry, thats opaque04:07
wallyworldobserver and restricted observer are granted access roles04:07
lifelesswallyworld: so is e.g. driver, and owner.04:07
wallyworldbut if you are a project owner, then you can see stuff04:07
lifelesswallyworld: for performance we almost certainly want the owner etc in the one system, with checks that they are in sync.04:07
wallyworldbut if you register a project, you are the owner without it being granted04:07
wallyworldie visibility is implicit in that case04:08
lifelesswallyworld: there is a difference between 'the user does not click to grant it' and 'it is not granted'04:08
wallyworldi know ownership can be assigned04:08
wgrantI think it should be the default, but it should not be implicit.04:08
wallyworldlifeless: agree on the difference04:08
wallyworldbut if i register a project, i want to implicitly be able to see everything by default04:09
lifelesswallyworld: its my understanding that owners etc won't be checked during lookup: if its not in the table, access denied.04:09
lifelesswallyworld: no, you want to see everything by default. The word implicit isn't needed for the use case :)04:09
lifelesswallyworld: I am asserting that your needs are met by:04:09
lifeless - defaulting project owner to full access (much like we default team owners to administrator status)04:10
wallyworldlifeless: the balsamiq mock ups imply implicit visibility due to project roles04:10
lifeless - possibly having a rule that the owner must retain full access04:10
wgrantOld mockups are not gospel :/04:10
wallyworldsure, but they are the current starting point04:11
wallyworldif we decide to do things differently so be it04:11
wgrantMm, I think they are hints.04:11
wgrantNothing more.04:11
wallyworldnow we have an interactive demo based on the mockups  which can be used to try and help clarify these sorts of requirements04:13
lifelessanyhow, my point is this: you are saying its complex to describe because you have multiple implicit accesses + granted access.04:13
wallyworldyes, that's my current understanding04:14
lifelessone way to address that, and a way I would encourage for reliability and performance in the system, is to drop implicit access. Instead use business rules to populate granted access when something should be automatic.04:14
wallyworldno problem with that, i agree04:14
wallyworldbut we may still want to distinguish to the user that is the case04:14
wallyworldie someone has access due to automatic propogation based on business rules04:15
wallyworldrather than an explicit grant of access by an authorative user04:15
lifelessthats an interesting case04:15
wallyworldso that if someone needs to be removed from the bug supervisors team, they ca nbe'04:16
lifelessI think that the authoritative user still exists (they added someone to the role)04:16
wallyworldwhat about the case where a team is assigned as bug supervisor.04:16
wallyworldand someone in the team should be cut iff04:16
wallyworldcut off04:16
lifelessand the grant is a consequence of being added to the role - but that doesn't imply revoking when removed from the role04:16
wallyworldbut not the whole team04:16
lifelesswallyworld: I don't think we should support that04:17
lifelesswallyworld: let me be more clear: supporting that will be extremely hard.04:17
wgrantYou can't just remove them from the team?04:17
lifelesswallyworld: either remove them from the team, or use a team that accurately models the supervisors desired.04:17
wallyworldwgrant: yes, but you need to see why and how someone has acess so you know what you need to do to remove visibility04:17
wallyworldlifeless: see above04:17
lifelesswallyworld: so in this situation,it would show 'user FOO has access because they are in team BAR which has FULL ACCESS'04:18
wgrantwallyworld: Sure, you'll see that they're a member of Ubuntu Bug Control, and that team is granted access.04:18
lifelessI presume there is a free text field related to each observer, for explanations ?04:18
wallyworldnot that i am aware, but i could be wrong04:18
lifelesse.g. 'fred is auditing us for iso9008:2000 status'04:18
wallyworldthat may be something to add04:19
wallyworldif it is not already there04:19
lifelessthat would supply a way to explain business rule added access too - 'BAR added when set as bug supervisor'04:19
wallyworldlifeless: with your example above, it may be that we want to say ".. in team BAR which has FULL ACCESS because team BAR is the bug supervisor"04:20
wgrantOf course, ideally all access control would be managed through this view...04:20
lifelessthat isn't enough to support automatically removing BAR when changing bug supervisor (but that may be undesirable anyway)04:20
wgrantSo it would show the bug supervisor privilege for that team as well.04:20
lifelesswallyworld: understood, I can't comment about whether we need that or not :)04:20
lifelesswallyworld: I will comment that for teams we started with implicit stuff for owners04:21
lifelesswallyworld: and have slowly, to great rejoicing, removed it.04:21
lifelesswallyworld: so my *hunch* is that it will be a problem.04:21
wgrantAs Curtis and I discussed last week about bug supervisors.04:21
wallyworldi agree 100% that the model will be much simpler to have one access lookup meachanism, and that owners etc should be put there via business rules04:22
wallyworldbut i think we need to record that, is all04:22
wallyworldso we can show it to the users04:22
wallyworldto explain *why*04:22
wallyworldsomeone has access04:22
wgrantYou might recall that we discussed this on maybe Wednesday's call. The thought was that ideally the roles would not be conflated at all, but the initial project setup wizard would guide you through setting probably the same people into each role.04:24
wgrantThe explanation only needs to exist if the roles are configured in separate places and somewhat conflated.04:24
wgrantIdeally they would be separated.04:24
wgrantAnd manageable from the same place.04:24
wgrantSo it's obvious what's what.04:24
wgrantRather than having the bug supervisor on one page, security contact on another, and then a huge list of (restricted) observers.04:25
wgrantAll separated.04:25
wallyworldyes, i recall. but relying on the setup wizard is not enough.04:25
wgrantWith no single view of access control.04:25
wallyworldthey would be separated04:25
wallyworldall users with access would be shown on one page04:25
wallyworldregardless of how they got that access04:26
wgrantI speak of more than just project-wide observer access04:26
wgrantThe same page really should list everyone with bug supervisor access, security contact membership, etc.04:26
wgrantIt probably won't in the first iteration.04:26
wgrantBut we should certainly not design ourselves into a corner.04:27
wallyworldyou mean for a given project though?04:27
huwshimi... and this is why I should just JFD things instead of asking questions on IRC :P04:27
wallyworldand that's what the mockups do now04:27
wgrantwallyworld: Hm?04:27
huwshimiwallyworld: Is "member" a confusing word to use?04:27
wgrantwallyworld: They list everyone *with observer access through those roles*.04:27
wgrantwallyworld: They don't take into account the privileges that those roles actually exist to grant.04:27
huwshimiwallyworld: Actually on second thoughts I guess it probably is04:28
wallyworldhuwshimi: i can't think of a better term now04:28
wallyworldwgrant: what page mockup are you looking at?04:28
huwshimiwallyworld: Better than "Observer"?04:28
wgrantwallyworld: None.04:28
wallyworldhuwshimi: maybe i don't have the context - you talking about team members?04:29
wallyworldwgrant: when you said "they list everyone.....", what does "they" refer to then?04:29
wgrantwallyworld: The last version of the mockups that I saw.04:30
huwshimiwallyworld: definitely confusing then... I'll try and come up with something else04:30
wgrantThey only deal with observer access.04:30
wgrantNot any other kind of access, like, say, bug supervisor write privileges.04:30
wallyworldwgrant: and restricted observer, and access via a project role04:30
wgrantThey're just different paths to the same privilege.04:31
wallyworldnot really? restricted observer != observer04:31
wgrantRestricted observer is the same sort of privilege, just over a smaller scope.04:31
wgrantThe privilege is visibility.04:31
wgrantThere are other privilegs.04:31
wallyworldlike ability to assign/triage bugs?04:32
wallyworldyou mean?04:32
wgrantWhich is what bug supervisor really means.04:32
wallyworldsure, but that stuff we didn't do for this first cut04:32
wgrantBut it's very relevant for if you want to describe why someone has a permission.04:33
wgrantBecause then their access management page will say that they are the owner and an observer.04:33
wallyworldyes, so there is a project role filter04:33
wgrantRather than just an observer because they are the owner.04:33
wallyworldi think we sort of agree04:34
wgrantSo, your explanation suggestion is only needed because A) I haven't convinced you yet that project roles have nothing to do with observer privileges, except for being sensible defaults during the setup wizard; and B) the UI that is in scope right now will only cover observer privileges.04:35
wgrantIf A becomes no longer the case, then "PRIVILEGE because ROLE" is no longer the case.04:36
wgrantIf B becomes no longer the case, then the team's access management page will list ROLE anyway.04:37
huwshimiwallyworld: So is an Observer part of a team or are they just directly connected to a project somehow?04:43
wallyworldhuwshimi: an observer is a person or a team who have been granted access to view a project's private artefacts04:44
wallyworldso a person may be an Observer via their team membership04:44
wallyworldhuwshimi: a Restricted Observer can just see private bugs and branches04:45
wallyworldhuwshimi: an Observer can also see milestones, blueprints etc etc04:45
wallyworldmake sense/04:45
lifelesswallyworld: thats temporary04:46
huwshimiwallyworld: But an Observer can also see a projects private artefacts without being part of a team that gives them that permission right?04:46
lifelesswallyworld: long term restricted will be able to be granted milestones blueprints etc04:46
lifelesswallyworld: there is no intention to arbitrarily limit the things restricted observers can be given access to see04:46
wallyworldhuwshimi: yes, a person may be individually made an observer04:47
huwshimiwallyworld: OK great thanks04:47
lifelesshuwshimi: or a team they are in (standard LP idiom there)04:47
wallyworldlifeless: i said that above :-)04:48
lifelessah ll04:48
wallyworldlifeless: i assume the long term then is to allow projects to define what 'restricted' means04:49
lifelesswallyworld: I don't know what you mean04:53
wallyworldlifeless: so you said "restricted = bugs and branches only" is temporary04:53
wallyworldand that later04:54
wallyworldrestricted would be gratned access to blueprints etc04:54
lifelessrestricted would *permit* granting access to blueprints etc04:54
lifelessthe limitation to bugs and branches is an implementation limit04:54
wallyworldyes, that's what I was asking, thanks04:54
wallyworldjust wanted to be sure i understood04:55
lifelessAFAIK there is no intent for restricted to ever change its meaning: you will always need restricted *to a concrete thing*04:55
lifelessits just that the things we can put in the concrete list will expand as we implement more04:55
wallyworldyes, that's how i understood it from what you said04:55
lifelesswe're synced then :)04:56
wallyworldfor now until more people start to discuss it :-)04:56
* wallyworld runs off tp pick up kid from school04:58
huwshimiwallyworld: haha04:58
lifelesscan has review? https://code.launchpad.net/~lifeless/oops-tools/nomoreprefix/+merge/7785805:07
lifelesswgrant: diff is there, if I can tempt you :)05:09
wgrantlifeless: prefix.upper?05:13
wgrantThat seems unnecessary, and will break eg. SSO.05:13
=== almaisan-away is now known as al-maisan
=== al-maisan is now known as almaisan-away
wgrantIt turns out that cloning a container works better if you don't have the MAC overridden in the config to a constant value.05:39
lifelesswgrant: its moved lines, it is necessary05:39
wgrantlifeless: Ugh, so it is. That's horrible.05:40
wgrantlifeless: Can we try using a sensible one first?05:41
wgrantThen fall back to the uppercase one.05:41
wgrantThen create one in sensible case.05:41
lifelesswgrant: thats a rabbit hole and orthogonal05:42
lifelesswgrant: e.g. searches probably do upper() at the moment.05:42
lifelessI'd love to fix oops tools up in general, but I don't fancy cascading tweaks when working on a different focus05:45
wgrantlifeless: Yeah, I get what you're doing now.05:50
wgrant(sorry, lightdm decided that sharing was caring, so it donated lots of my keystrokes to the underlying getty)05:50
lifelesswgrant: I take it my cover letter wasn't helpful enough ?05:53
wgrantlifeless: I don't understand your req_vars and statements changes in models.py, or your second hunk in oops.txt.05:55
wgrantWas it creating a duplicate QSMMX?05:56
lifelesshigher up in oops.txt it calls load_oopses05:56
lifelessand loaded them all05:56
lifelessthe sameple QSMMX oops was previously silently ignored (due to the prefix not existing)05:56
wgrantOf course.05:56
lifelessso this change makes it get loaded by that call (line 8 I think)05:56
lifelessthe req_vars and statements changes are future proofing for oopses from scripts that don't have req_vars, or statements.05:56
wgrantSo they're unrelated?05:57
lifelesssnuck in by testing with a minimal plausible oops.05:57
wgrantThought I must have been missing something.05:57
lifelessnah, I failed to write that up. mea culpa05:57
wgrantOh right, HA librarian tonight.06:06
lifelesswant something scary?06:09
wgrantI am terrified.06:09
lifelessoops-tools will discard oopses if rsync write them to disk out of order & loading runs during the rsync06:09
lifelessout of order being what ext3 dirindex does, of course.06:09
wgrant"discard" meaning "not see", or actually delete?06:10
lifelessnot load06:10
lifelessif there are 10 oopses when it finishes a run, but they are not range(1,11), then when it backfills, the optimiser which records len() will skip the early numbers06:11
lifelessand this is why I was looking at getting set data from the db :)06:11
=== almaisan-away is now known as al-maisan
* StevenK hopes his last ec2 run finishes quickly.07:46
StevenKGiven it started first, it's a little worrying.07:46
* nigelb wonders if buildbot will also be public.07:48
StevenKI'm hoping for our CI infrastructure to be public, be it buildbot or jenkins.07:51
StevenKBut I'd prefer the latter.07:52
bigjoolsgood morning all08:45
nigelbGood morning bigjools08:46
lifelessbigjools: 'lo08:46
bigjoolshowdy hackers08:47
lifelessbigjools: hey, so what was wrong with that backported option ?08:48
bigjoolslifeless: it's awful08:48
bigjoolsfor starters, referring to python functions on a command line option is evil08:48
lifelesswith you so far08:49
bigjools2nd, you have to hard code the log name in the function. I wrote code to work around that but again it's evil08:49
lifelessurgle, so you can't pickup the twistd option normally?08:49
bigjoolsin the end --logfile=/dev/null is much easier :)08:49
bigjoolsnope, remember that there's 2 sets of options08:50
bigjoolsone for twisd, the others for the plugin08:50
lifelessbigjools: ah, headdesk. Separately... I closed the bug you filed, but wanted to touch base with you to make sure you understand why08:50
bigjoolswhich bug?08:50
lifelesson oops-twisted08:50
bigjoolsnot read my email yet08:50
bigjoolsoh, the fallback08:50
bigjoolsI must admit, that way of doing it surprised me08:50
lifelessbecause twisted's log mechanism sends to all observers, having one observer be a log file and another be the oops system causes failures to be logged twice - once in the log as a default thing, and once as an oops08:51
lifelessso you have to have only one observer - the oops one, and have it *translate* the failures for logging as 'oops id XXX created'08:51
bigjoolsthat's not how I expected it to work either08:52
bigjoolsI was expecting that the oops observer would *only* add the oops info08:52
bigjoolswhy does it need to translate?08:52
lifelessYeah, I can see how you might expect that - but can you see how its constrained by the broadcast setup they have ?08:52
bigjoolsnot really :) but I am listening08:53
lifelessok, so the default is to log the backtrace08:53
lifeless30 or 40 lines or whatever.08:53
lifelesswe don't want that, we want to log 'created oops id XXX ValueError: foobar'08:53
bigjoolswhy don't we want that?08:53
lifelessbecause it destroys readability of the log, and the exception and all its data is in the oops08:53
bigjoolsI'd say that was subjective, but ok, it would be consistent with what LP does already08:54
lifelessof course, if oops reporting is disabled, we want the traceback08:54
lifelessI'll agree that its subjective08:55
* bigjools 's inner KDE user is screaming "make it configurable!"08:55
bigjoolsanyway it all works hunky dory now08:56
lifelessok great08:56
bigjoolsif we get time we'll add another buildout recipe to wrap all this up into a nicer single script08:56
bigjoolssince the "--logfile /dev/null txlongpoll" is always required08:57
lifelessheaddesk. We really should lock down old fixed bugs.08:58
lifeless(bug 7 comment..)08:58
_mup_Bug #7: Need help for novice translators <lp-translations> <Launchpad itself:Fix Released by danilo> <Launchpad Documentation:Fix Released by matthew.revell> < https://launchpad.net/bugs/7 >08:58
bigjoolsat least it's not pornspam09:08
nigelbalthough the more fun one was  - "Dear bug, I'd like to add you to my network"09:09
lifelesswgrant: care for a follow on review to the prior ?10:14
lifelesswgrant: https://code.launchpad.net/~lifeless/oops-tools/outoforder/+merge/7788710:14
lifelessthe diff is there, the warning is bogus10:14
wgrantlifeless: Not right now, sorry.10:15
lifelesswho else is around, not on leave, and a reviewer....10:16
=== al-maisan is now known as almaisan-away
lifelessoh look, stub.10:16
stublifeless: private branch?10:17
lifelessstub: private project, pending some feedback I've asked for today from various folks10:17
lifelessstub: open sourcing it should happen soon, when we will blind anyone that reads it ;)10:18
lifelessthink men in black, done via the code.10:18
lifelessstub: you should have access though10:20
bigjoolslifeless: your change to the title on https://bugs.launchpad.net/launchpad/+bug/297471 is not English!10:23
_mup_Bug #297471: PPA upload notifications always and only go to the signer <email> <lp-soyuz> <oem-services> <ppa> <Launchpad itself:Triaged> < https://launchpad.net/bugs/297471 >10:23
* bigjools fixorates10:25
wgrantbigjools: It was fine!10:34
stublifeless: Do we cope with partial OOPS files now, such as an rsync in progress or being killed midrun?10:34
wgrantbigjools: They always go to the signer, and they only go to the signer.10:34
bigjoolswgrant: well I misread it a few times, so no it wasn't10:34
bigjoolsthe English is awkward10:35
lifelessstub: rsync does a tempfile + rename10:35
lifelessstub: so we ... probably do see the tempfile, which will barf a oops parsing error, which is silently caught and discarded10:36
stubWe might do it that way...10:36
nigelbwgrant: I just found out the reason for Ubuntu Mono being smaller :|10:36
wgrantnigelb: Oh?10:36
stublifeless: Yer, but now when we rescan the file that previously barfed should complete.10:36
nigelbwgrant: bug 72773310:36
lifelessstub: yes10:36
lifelessstub: it takes 3.5 seconds to get 10K rows (for a moderately busy day, one appserver) out of the db on carob10:37
stubWhich we do every 10 minutes, as that is how often the files get synced10:38
wgrantnigelb: Hmm.10:38
lifelesswell, no, there is a cronscript, and it walks all active (cronscripts + appservers) at once10:38
lifelessbut -most- dirs are much much much smaller10:38
lifelessand a rabbit injector won't need to walk dirs at all10:39
lifeless(which is what this is all leading up to)10:39
nigelbwgrant: Launchpad now looks much smaller and there's no clean way I can think of to fix it that falls back gracefully.10:39
wgrantnigelb: Yeah, it's pretty small :(10:39
nigelbby 2 pts! :(10:40
wallyworld_stub: there was going to be a restore of the qas db from prod over the weekend i think. Do you know if it succeeded?10:52
wgrantwallyworld_: It did.10:52
wgrantwallyworld_: I finished it off with bradm this morning.10:52
wallyworld_wgrant: cool. i'll land that db patch10:53
wgrantIt is now nice and shiny from a Thursday or Friday backup.10:53
stubIf landing a db patch was blocked on refreshing production data, does that mean we are not running garbo jobs on qastaging? Or was it a manual fix of production data?10:53
wallyworld_stub: we were running garbo but i landed a branch to delete it a bit too soon. cause i didn't realise we ran garbo on qas10:55
wallyworld_i thought we used a db restore to update qas10:55
wallyworld_i think until recently we did indeed not run garbo on qas?10:55
stubWe do use a db restore to update qas. If we are not running garbo jobs on qas, we should be.10:56
wgrantWe do run them on qas.10:56
wallyworld_stub: we are running them now. but i thought we weren't10:57
wallyworld_i *think* it was a recent server upgrade which know allows us to10:57
stubI dunno what was.10:57
wallyworld_or at least that's what i think i was told10:57
wgrantgarbo's been running for a while.10:58
wgrantBut most other stuff doesn't.10:58
lifelessstub: thanks!11:09
=== almaisan-away is now known as al-maisan
bigjoolswgrant: I am starting to look at bug 34086, if you have any thoughts already let me know11:49
_mup_Bug #34086: removal of arch-all packages while there are arch-specific packages dependent on it results in uninstallable binaries <escalated> <feature> <lp-soyuz> <rls-mgr-o-tracking> <soyuz-publish> <Launchpad itself:Triaged> < https://launchpad.net/bugs/34086 >11:49
wgrantbigjools: Urgh.11:50
bigjoolswhen I say "starting to look" I mean evaluate what can be done11:51
wgrantThis is pretty related to the sources thing.11:51
wgrantWe should probably approach them in similar ways.11:51
bigjoolsthe key part is working out what the dependent packages are11:52
wgrantThere are two ways.11:52
wgrantI mean, two ways to approach this.11:52
wgrant1) Don't remove packages if something is dependent on them.11:53
wgrant2) Remove all binaries for a source and arch at once.11:53
wgrantMethod 1 is just about guaranteed to drive you to insanity and terrible, terrible code.11:53
wgrantAnd terrible, terrible cruft.11:53
wgrantAnd won't catch other uninstallability cases.11:54
wgrantSo is mostly pointless.11:54
wgrant2 is easy, has been listed as a TODO since 2005, and I believe it fixes the important cases here.11:54
bigjools"Need to make binaries go in groups" is the comment in the code :)11:54
wgrantSo, talking about dependent packages is probably going to lead you nowhere good.11:55
wgrantBest to ignore that line of investigation, I think.11:55
bigjoolswell, no11:56
bigjoolsI am considering all avenues, we'll use the best one11:56
wgrantWell, I consider 1 to be not much better -- but significantly harder -- than 0) Don't ever remove anything at all.11:57
bigjoolsyou don;t understand my point11:57
bigjoolsanyway, lunch calls11:59
=== benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: benji | Critical bugs: 253 - 0:[#######=]:256
nigelbIs jtv off/away today?12:30
jelmernigelb: Yep12:31
nigelbaha, that explains that12:31
rvbaHi benji.  Would you be kind enough to mentor 2 of my reviews? (I've done the first branch and I'm currently reviewing the second one)12:52
benjirvba: sure, what can I do fo you?12:53
rvbabenji: Thanks, here is the first one: https://code.launchpad.net/~allenap/launchpad/longpoll-associate-consumer-not-now/+merge/7789412:54
jelmerbenji: hi13:15
benjihi jelmer13:15
jelmerbenji: I've got a fairly long merge proposal that's already been reviewed by a couple of others. I think I've now adressed all issues that were raised, can you have another look at it?13:19
benjijelmer: sure13:19
jelmerit is https://code.launchpad.net/~jelmer/launchpad/bzr-code-imports-ui/+merge/7315313:19
jelmerbenji: great, thanks :)13:19
rvbabenji: Here is the second review if you still have time to mentor it https://code.launchpad.net/~allenap/launchpad/longpoll-finish-read-only-request/+merge/7791513:56
benjirvba: sure13:56
allenaprvba, benji: Thank you for the reviews :)14:36
=== al-maisan is now known as almaisan-away
deryckabentley, I saw you marked bug 758896 OPINION.  Have we started using this status for ourselves now?15:00
_mup_Bug #758896: Checklist items and flashes send mixed messages <exploratory-testing> <upstream-translations-sharing> <Launchpad itself:Opinion> < https://launchpad.net/bugs/758896 >15:00
abentleyderyck: I think OPINION describes the status better.  When jml was product strategist, his opinion had more weight, but I think he was wrong then, too.15:01
deryckabentley, well, OPINION is a closed bug though.  I don't think this bug should be closed.15:02
deryckabentley, OPINION says, "closing this bug acknowledging we have a difference of opinion."  I don't think you mean that, do you?15:03
abentleyderyck: No, I meant "We have a difference of opinion about whether this is a bug".15:03
abentleyderyck: The implication that this is a closed bug escaped me.15:04
deryckabentley, hmmm, so you're reading of the status isn't far off, but I think it being closed means it shouldn't be an opinion bug.15:04
abentleyderyck: Are you saying the opinion status shouldn't imply a closed bug?15:05
deryckabentley, no, I'm saying opinion status shouldn't be used here because it is a closed bug.15:05
derycksorry, I wasn't clear15:05
abentleyderyck: it was a triaged bug, not a closed one.15:06
deryckabentley, again, not being clear, sorry. :)  please change it back to triaged so the bug will not be closed. :)15:07
deryckI don't think we should close this bug.15:07
abentleyderyck: I follow.15:07
deryckabentley, but I wanted to understand your thinking and not come along and change it back without chatting with you about it.15:08
abentleyderyck: ISTM that opinion is actually neither open or closed.  It is "closed to some, open to some, depending on their opinion".15:10
deryckabentley, yes, no one is clear on what OPINION means as a status.15:12
jelmerI've only seen "Opinion" being used as a polite format of "Won't Fix".15:12
abentleyjelmer: That kind of applies here.15:14
benjijelmer: I just finished with https://code.launchpad.net/~jelmer/launchpad/bzr-code-imports-ui/+merge/73153.  I included a lint report for you.15:19
jelmerbenji: Thanks!15:19
jelmerbenji: any chance you can repeat your comment and vote at https://code.launchpad.net/~jelmer/launchpad/bzr-code-imports-ui/+merge/77961 ? I resubmitted the MP while you were reviewing, it seems (no new revisions).15:20
benjijelmer: done15:22
jelmerbenji: Teh awesome. Thanks again.15:23
deryckHi, benji.  I have a branch for review if you're available.15:29
benjideryck: sure15:30
deryckbenji, thanks!  https://code.launchpad.net/~deryck/launchpad/target-series-bug-834082/+merge/7796615:30
=== deryck is now known as deryck[lunch]
deryck[lunch]benji, I'm lunching early but here if you have questions about that branch.15:36
benjideryck[lunch]: ok15:36
jcsackett_sinzui: you around/free to chat?15:42
sinzuiI am15:42
benjideryck[lunch]: your branch looks good15:43
deryck[lunch]benji, awesome, thanks!15:44
sinzuijcsackett_, https://bugs.launchpad.net/launchpad/+bug/86546715:47
_mup_Bug #865467: bug task missing from affects table because bug supervisor is not subscribed <disclosure> <regression> <Launchpad itself:Triaged by jcsackett> < https://launchpad.net/bugs/865467 >15:47
sinzuijcsackett_, bug 375331 is a similar permission issue. bug 113825 describes the ambiguity in the ui. I doubt the later is in scope, but it does help explain the confusion about this issue16:17
_mup_Bug #375331: Some extra filebug options don't show up for projects not having a bug supervisor <bugs> <projects> <Launchpad itself:Triaged> < https://launchpad.net/bugs/375331 >16:17
_mup_Bug #113825: Null bug supervisor is confusing <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/113825 >16:17
=== deryck[lunch] is now known as deryck
=== beuno is now known as beuno-lunch
barrygary_poster: ping17:59
=== beuno-lunch is now known as beuno
nigelbIts not that late yet.18:34
nigelblifeless: early start?18:34
lifelessa little late18:35
* nigelb hasnt seen 6 am to 8 am in a while :P18:35
=== jcsackett_ is now known as jcsackett
gary_posterbarry, hi!18:52
barrygary_poster: hi!  i *think* i've answered my own questions, re: autoppa and doing a storm release.  or let's say, i've worked around my issues :)18:52
gary_posterbarry, ok cool-ish :-)  sorry wasn't around18:53
barrygary_poster: no worries.  at least it's python code so i can always trace what it thinks it should be doing and then make it dtrt :)18:53
gary_poster:-) cool18:53
lifelessflacoste: hi; call timeish ?19:37
lifelesssinzui: medium bugs down to < 1 page19:39
flacostelifeless: sure19:45
flacostelifeless: skype me when ready19:50
mwhudsonhm, bin/retest appears not to work with ec2 mails any more?20:14
=== al-maisan is now known as almaisan-away
=== benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 253 - 0:[#######=]:256
* mwhudson smacks head again22:15
mwhudsonwhy do the feature flag context manager thingies install a new feature controller?  afaics it's only because of the caching22:15
mwhudsonthat FeatureController does22:15
cjwatsonhmm, so I was wondering about an issue relating to writing the Debian->Ubuntu mass-sync API script on top of the new Archive.copyPackages method22:41
wgrantWhat in particular concerns you?22:42
cjwatsonwhen we did this with 'sync-source.py -a', the syncs showed up as having been performed by "Ubuntu Archive Auto-Sync <archive@ubuntu.com>"22:43
cjwatsonthis was generally a good thing - crediting to the admin who performed the auto-sync would give them way too much credit and end up with people assuming they knew things about particular packages and such22:44
cjwatsonI'm wondering how we can do that with the new method, given that:22:44
cjwatson(a) bug 827555 isn't done yet22:44
_mup_Bug #827555: native syncs have no way to indicate sponsorship <derivation> <Launchpad itself:Triaged> < https://launchpad.net/bugs/827555 >22:45
cjwatson(b) the ~katie user apparently no longer exists?22:45
wgrantkatie is suspended. Still exists, but can't be browsed to.22:45
StevenKAnd she is still a celebrity, right?22:45
cjwatson(c) I'd prefer auto-syncs to be performable by any archive admin without having to pass around credentials for an LP account22:45
wgrantStill a celebrity, unfortunately, yes.22:45
cjwatson(yes, I should have thought of all this ages ago)22:46
wgrantcjwatson: Indeed. I don't think Red was really aware of how syncs are used.22:46
cjwatsonI don't want to apportion blame ...22:46
wgrantSome could say that this is a problem that comes from having random people develop software like this :P22:46
cjwatsonI'd rather solve the problem22:47
lifelesscjwatson: so, one option is to implement (a)22:47
wgrantI think that's the best option.22:47
lifelesscjwatson: I suspect its tractable22:47
cjwatsonif people think that's the best option, and if it doesn't have too many tentacles, I'd be happy to have a go22:47
cjwatsonI wouldn't mind advance advice on the likely tentacles, though22:47
lifelessavoid the blue ones, they itch.22:48
cjwatsonso we'd then basically do copyPackages(person=lp.people['katie'], ...), yes?22:48
lifelessso things to think about22:48
lifelessshould there be limits on who you can credit ?22:49
cjwatsonThe last time I thought about that, I decided it wasn't a problem since any uploader can already write random crap into the changelog trailer line if they like, and this should be just the same.22:49
lifelesscjwatson: I think you'd want to lookup the person by email rather than hardcoding katie in there.22:49
cjwatsonI was abbreviating, but OK :)22:49
* StevenK kicks HE and Fremont 1.22:50
wgrantStevenK: v6 is working fine.22:50
lifelessuhm, you will need to care for backwards compat when you change the job metadata (because we don't rollout atomically)22:50
wgrantv4 seems to still not have routes.22:50
StevenKwgrant: Indeed.22:50
StevenKWhich is how I'm connecting.22:50
wgrantBut v6 was down for ages too :(22:50
cjwatsonlifeless: sorry, is that to me?22:50
lifelessother than that, I presume the output fields already exist22:50
lifelesscjwatson: yes22:50
lifelesscjwatson: jobs run via a json blob of instructional metadata22:50
wgrantSo, this whole thing is pretty awful.22:50
cjwatsonI hadn't got as far as looking at the existing schema22:50
wgrantBecause it's only partially implemented.22:51
StevenKPCJs is .. sort of horrible22:51
wgrantBuild notifications etc. still only take the SPR's properties into account.22:51
StevenKBut it's better than what we had before.22:51
wgrantIt's not.22:51
wgrantIt's the same as we had before, just for the third time.22:51
lifelesscjwatson: that persists in the db between submission and completion, so changes to the code need to assume (A) old submissions are yet to be processed and (B) new submissions may get picked up by an old processor22:51
cjwatsoncompat both ways, ok22:51
StevenKAt least it isn't delayed copies madness.22:52
lifelesscjwatson: does the output field you want to put the sponsor into already exist ?22:52
lifelesscjwatson: if so I think that that is the entire list of tentacles22:52
StevenKI doubt it.22:52
cjwatsonit should do22:52
wgrantIt doesn't.22:52
wgrantIt's not clear what the sponsor actually means.22:52
cjwatsonI think it should act the same as citing somebody else's name in the changelog and then signing it yourself22:52
StevenKThere is requestor, but not sponsor22:52
wgrantWhat does it mean? :)22:52
lifelesswell, how does the older sync do its job ?22:52
cjwatsonwhy isn't that the same?22:52
wgrantcjwatson: Heh heh heh.22:52
wgrantcjwatson: That's implemented as properties of the SPR.22:53
cjwatsonlifeless: the older sync is admin-only and can pretend to be whoever it likes22:53
wgrantSPR.creator and SPR.signing_key22:53
cjwatsonwgrant: oh right22:53
cjwatsonand of course now we have the same SPR as Debian22:53
wgrantcjwatson: This should have been fixed as part of DDs, but was conveniently swept under the carpet.22:53
wgrantSo much so that until last week we didn't even track who did syncs.22:53
cjwatsonbugger.  then this is actually quite hard.22:53
wgrantYes, it needs new modelling.22:53
lifelessok, so a schema change as well, and a little code to treat NULLs there as the current behaviour. New uploads to set the field, and the sync change to let it be controlled.22:53
wgrantSchema changes are cheap now, but we need to actually think about it.22:54
cjwatsonI might just have to keep using sync-source.py -a for a while in P22:54
lifelesscjwatson: doing the schema change is now straight forward22:54
cjwatsonwhich I'd hoped not to do, but isn't the end of the world22:54
wgrant(and because it wasn't done as part of the project, we can't now have a simple, coherent model... we have to hack this together on top of existing crap :D)22:54
StevenKIf it's just a JSON field, it requires no schema change.22:54
wgrantStevenK: We can record it in the job.22:54
wgrantBut that's all.22:54
cjwatsonlifeless: right, but trying to do data remodelling at the same time as an Ubuntu release may be a little beyond my cognitive capacity22:54
wgrantAnd recording it in the job isn't very useful.22:54
StevenKHopefully having MOTUs and core devs do their own syncs hasn't been horrible.22:57
cjwatsonthat's been largely fine22:57
wgrantStevenK: Depends on your definition of horrible...22:57
wgrantIt seems to be working OK now.22:57
cjwatsonsome teething trouble but the basic facility is great22:57
wgrantAnd we even have some form of auditing as of a week ago.22:57
cjwatsoncan't switch over completely to it until sponsorship is there though22:57
* cjwatson gets dragged off by toddler, brb22:58
wgrantsinzui: No call?23:02
jelmer_has anybody seen spurious failures in lib/lp/registry/browser/tests/poll-views_0.txt ?23:03
wallyworld_sinzui: wgrant: unity issues, be on call soon23:03
wgrantjelmer_: I haven't.23:03
sinzuimy audio appears to be knackered again23:05
wgrantsinzui: I think we heard you for a moment.23:05
StevenKI did.23:05
jelmer_wgrant: guess it might be something else, I'll look further23:06
=== Guest16725 is now known as StevenK
sinzuiStevenK, 85371023:57
sinzuiStevenK, bug 85371023:57
_mup_Bug #853710: Teams that have ever had a PPA cannot be renamed <Launchpad itself:Triaged> < https://launchpad.net/bugs/853710 >23:57

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!