StevenKsinzui: It is not an error if the DSP has no publishing_history00:07
sinzuiStevenK, Really? How can it be legitimate if it has not even gotten to the pending state. How do we ensure that https://launchpad.net/ubuntu/+source/html5-browser (ubuntu/html5-browser) is not accepted as an Ubuntu package?00:58
wgrantMy latest branch touches on that for bug targetting.00:58
wgrantBut only in terms of erroring, not restricting vocab results.00:59
sinzuilifeless, I never want to see an open-team membership notification. I was tempted this weekend to hack a fix for team mail notification so that I can have a rule to delete them00:59
sinzuiI will see that in a few minutes then00:59
StevenKsinzui: Hmm. You may be right. I will investigate today.01:00
lifelesssinzui: https://bugs.launchpad.net/launchpad/+bug/81562301:00
_mup_Bug #815623: Mail notifications sent to team admins on joins / leaves to open teams <Launchpad itself:Triaged> < https://launchpad.net/bugs/815623 >01:00
lifelesssinzui: I have a branch that nukes the notifications01:00
lifelesssinzui: it has one wart01:00
StevenKBut it looks like some poeple want the notification01:00
lifelesssinzui: it squelches notification about folk that become / leave the admins of the team, which is still actually restricted; I think thats undesirable but not a Big Deal01:00
sinzuilifeless, yes. that is a bad01:01
lifelesssinzui: so I'd like your input on two things01:01
lifelesssinzui: does the admin changes not being notified matter ?01:01
lifelesssinzui: -and-01:01
sinzuilifeless, this wart is one if underlying issue team notifications were not an easy fix.01:01
lifelesssinzui: do we care about the folk that care about these spam messages ?01:02
lifelesssinzui: I can fix it to retain admin notifications by checking the status field of the teammembership change, I suspect.01:02
lifelesssinzui: but I don't know if we need to do that, or if we can file a high bug to come back and reinstate just those messages.01:03
sinzuilifeless, I am not aware of a team admin that wanted the those notifications. I would not want to put a switch in. I think this class of issue is really structural subscriptions for team, which is a request of sorts in our bug tracker already01:04
lifelesssinzui: so you support - squelch the notifications; when we do explicit subscriptions to team changes, then we can default a subscription for non-open team admins, default to no subscription for open team admins, and moderation etc will still be automatic ?01:05
sinzuilifeless, yes01:05
lifelessdo you happen to know the bug # for subscriptions-to-team-changes ?01:06
sinzuiI can find it quickly01:07
sinzuilifeless, bug 50751501:08
_mup_Bug #507515: want to subscribe to changes in a team <email> <lp-registry> <teams> <Launchpad itself:Triaged> < https://launchpad.net/bugs/507515 >01:08
sinzuilifeless, you may be fixing the root cause of bug 28541401:08
_mup_Bug #285414: There should be an option so that team admins don't receive notifications <email> <feature> <lp-registry> <teams> <Launchpad itself:Triaged> < https://launchpad.net/bugs/285414 >01:08
lifelessI will comment to this effect in the bug, file a bug about no admin notifications on open teams, and land my branch.01:08
lifelessI think I am01:10
StevenKlifeless: I just think landing your branch will lead to more bugs along the lines of "I used to be notified, now I'm not."01:11
lifelessStevenK: which we'll dup onto bug 50751501:12
_mup_Bug #507515: want to subscribe to changes in a team <email> <lp-registry> <teams> <Launchpad itself:Triaged> < https://launchpad.net/bugs/507515 >01:12
StevenKFairy nuff01:12
lifelesssinzui: can admins make someone else an admin ?01:26
sinzuino. I can find the bug.01:27
lifelessthats ok01:27
lifelessit means that this notification change w.r.t. admins is ok01:27
lifelessbecause there is also no action the admins can take01:27
lifelessin fact, we shouldn't be notifying admins in that case, we should be notifying the owner.01:27
wgrantThey can demote others. But that's it.01:27
lifelesssinzui: https://bugs.launchpad.net/launchpad/+bug/81619601:30
_mup_Bug #816196: new / deactivated admins of teams are not notified to the owner <Launchpad itself:Triaged> < https://launchpad.net/bugs/816196 >01:30
lifelesssinzui: and https://bugs.launchpad.net/launchpad/+bug/815623/comments/801:30
_mup_Bug #815623: Mail notifications sent to team admins on joins / leaves to open teams <Launchpad itself:Triaged> < https://launchpad.net/bugs/815623 >01:30
lifelesssinzui: you might want to add tags to the new bug, I'm not sure what it should have01:39
sinzuilifeless, done: this is my short list of team email issues: https://bugs.launchpad.net/launchpad/+bugs?field.tag=teams+email&field.tags_combinator=ALL01:46
lifelesssinzui: surely we've fixed https://bugs.launchpad.net/launchpad/+bug/120037 as a drive-by by now ?01:52
_mup_Bug #120037: Action reported in email as done by owner rather than administrator <email> <lp-registry> <teams> <Launchpad itself:Triaged> < https://launchpad.net/bugs/120037 >01:52
poolielifeless: hi01:53
lifelesshi poolie01:53
pooliewhy use a pretty printer at all, rather than just having recommendations for code style?01:53
lifelesspoolie: I thought I covered that earlier in the thread; because its easier for contributors.01:54
pooliehm maybe i misunderstood01:55
pooliei think bouncing changes because they're not pep8 clean raises the bar01:55
sinzuilifeless, maybe. I thought there were one or two emails that were hard coded to admin01:56
pooliewould using a pretty printer make things easier?01:56
lifelesspoolie: we could eliminate all code review formatting discussion01:56
lifelesspoolie: that seems easier to me.01:56
poolieif it happens at pqm post-commit, and if the contributor uses 200-character wide lines, reviewers will still complain01:57
poolieor tell them to run the formatter themselves01:57
poolieoh, i guess that was mentioned01:58
lifelesspoolie: are you against the use of a pretty printer ?02:03
poolieonly slightly, because it seems to have some risk of breaking things and i don't think it gains very much02:05
pooliei am definitely in favor of making it easy to contribute, and in particular avoiding unnecessary rejections02:05
sinzuilifeless, I am not sure that bug is fixed. There are rules to fallback to an historic membership.reviewed_by.02:08
sinzuilifeless, NM. I think this issue was in TM.setStatus() and enforce its use now. I marked the bug released.02:12
lifelesssinzui: \o/ cheap bug fixes ftw02:13
=== almaisan-away is now known as al-maisan
lifelessdoes the API show hidden messages ?04:02
mwhudsoni wonder how many hours a year the buildds spend "Building database of manual pages ..."04:12
mwhudsonit's about 25% of the pbuilder run time on my machine...04:12
lifelesswgrant: StevenK: either of you know why we do this - https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-2032CD17#longstatements04:19
lifelessthe packagebuildjob query04:19
lifelessI meant04:20
wgrantlifeless: To show the failed/successful build counts, I guess.04:20
lifelesswgrant: I'm unclear what you mean in https://bugs.launchpad.net/launchpad/+bug/711073/comments/304:21
_mup_Bug #711073: Archive:+packages (Archive:+index) timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/711073 >04:21
lifelessare you saying 'the queries in this bug are fixed but others are a problem'04:21
wgrantlifeless: +index and +packages were 99% < 5s.04:21
wgrantNot fantastic, but not terrible.04:22
lifelessand bug 758258 is in-progress with no assignee ?04:22
_mup_Bug #758258: buildfarmjob schema is inefficient for reporting <timeout> <Launchpad itself:In Progress> < https://launchpad.net/bugs/758258 >04:22
lifelesswgrant: did you land the new columns & garbo jobs?04:24
wgrantNo. Was going to do it soon before the DB deploy to minimise conflicts, but then got unexpectedly moved to Disclosure when Yellow was prematurely moved to maintenance.04:24
wgrantThere's only a couple of code changes needed in addition to the DB changes.04:25
wgrantMost of them landed two months ago.04:25
lifelessso, new world order, schema and code separate ;)04:26
wgrantI never planned to land them together.04:27
lifelessbug https://bugs.launchpad.net/launchpad/+bug/81623304:27
_mup_Bug #816233: Archive:+packages timeouts in buildfarmjob status summaries <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/816233 >04:27
wgrantThey did not need to be.04:27
wgrantActually flattens SPRB and BPB.04:27
wgrantAnd mostly works.04:28
* wgrant links it to the bug.04:28
lifelessedge needs to begone04:29
wgrantAh, already is linked.04:29
wgrantWe should do that.04:29
wgrantOr just hit people.04:29
lifelesslosa are chewing through things now04:30
lifelesswhich reminds me04:30
lifelessjtv: around ? I'm guessing too early.04:36
lifelessstub: I've figured out one of our timeouts... due to a unique constraint.04:40
_mup_Bug #816235: Bug:EntryResource:linkBranch timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/816235 >04:43
lifeless(I've added a second comment there now)04:44
lifelesspoolie: hi04:57
pooliehi hi04:57
lifelessin bug 721166 yesterday04:57
lifelessyou pushed a branch04:57
lifelessand linked it to the bug04:57
_mup_Bug #721166: Tests sometimes fail on EC2 due to _LockWarner garbage <build-infrastructure> <spurious-test-failure> <Launchpad itself:In Progress by mbp> < https://launchpad.net/bugs/721166 >04:57
lifelessthe link timed out on you04:57
lifelessdid you retry it, or had you done a commit --fixes lp:721166 ?04:58
poolieistr getting a javascript error while establishing the link04:58
pooliewith no details04:58
lifelessit was OOPS-2032F2404:59
pooliewhy, did it leave debris behind?04:59
pooliehell orobot?04:59
lifelessI'm trying to confirm a theory behind the cause04:59
lifelessif I'm right, we may have an explanation for a wide range of sporadic failures, and more data on why backend scripts with long (>1 second) transactions are bad)04:59
lifelessthe other possibility is that the scanner updates the branch row... that may be it05:00
pooliei actually wondered whether to mention it and then decided that "i got a js error with no oops" was boring05:00
poolieok; can i do anything to help settle it?05:01
lifelessI've got it sussed I think05:03
_mup_Bug #816235: Bug:EntryResource:linkBranch timeouts due to branch scanner transaction length <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/816235 >05:03
poolieit reminds me a bit of some other bug i filed05:03
pooliei forget the number05:03
pooliewhere what should have been a trivial action timed out after 9 seconds05:03
lifelessI've been of the opinion that backend scripts are a problem for a while05:06
lifelessbut 'contention' was never a very satisfying explanation05:07
pooliei can't actually find that05:07
lifelessand I can hardly expect to get a cultural change to something that isn't quite as easy on the basis of something that isn't satisfying :)05:07
poolieis there {waves hands} a way to run the branch scanner at a different isolation level or something?05:08
lifelesswouldn't help05:08
lifelessthis is because we have foreign keys05:08
lifelessthe appserver commit cannot progress until it obtains a lock preventing anyone deleting or changing the id on the branch row.05:10
lifelesspg does row locks, I don't think it does row,column locks05:11
lifelessthere are some design things we can do05:11
lifelessputting status-tracking data in separate tables, for instance05:12
lifelessbut the biggest is short transactions05:12
wgrantUbuntuSourcePackageNameWidget :(05:12
wgrantCan I add a precommit hook forbidding any diffs matching [Uu]buntu?05:12
lifelessyou need a cultural change I think05:13
lifelessor rather, yes, if you give me the db one for pqm at the same time ;)05:13
poolieanyhow, the things i saw, of should-be-tiny transactions timing out, would be consistent with that kind of long running lock holding transaction05:13
poolieperhaps also with other things like a thrashing db, but in that case i presume we'd notice05:14
wgrantAnybody up for reviewing two ~100 line branches?05:15
wgranthttps://code.launchpad.net/~wgrant/launchpad/transitionToTarget-validate_target/+merge/69189 and https://code.launchpad.net/~wgrant/launchpad/launchpadtargetwidget-prefix-respect/+merge/6918705:15
stublifeless: another one for the 'long transactions are evil' bucket.05:30
lifelessyes, exactly05:30
lifelessthat and DRI :P05:30
stublifeless: not sure why the branch scanner needs long transactions. It could probably run in autocommit mode even, provided it remembered to insert rows in earliest -> latest revision order.05:33
wgrantIt already partially commits sometimes.05:34
wgrantBecause if the initial scan fails it needs manual cleanup.05:34
lifelessit probably updates the last_scanned field too early05:36
lifelessor something05:36
lifelessstub: how hard would it be to make the transaction reaper kill *updating* transactions over 2.5 seconds long ?05:43
wgrantGoodbye bug modifications.05:44
stublifeless: We could kill transactions that have been trying to execute an UPDATE for longer than 2.5-3.0 seconds (0.5 seconds range, as the field we need is only updated every 500ms). Not sure if that would help anything though.05:48
stublifeless: We can't kill transactions that issued an update earlier in their transaction and might be blocking others.05:48
lifelessstub: because idle in transaction might be a non-mutating request ?05:49
lifelessstub: we could introspect pg_locks ? <evil face>05:49
stubBetter off doing all this with query timeouts.05:50
lifelesswas really just speculating05:50
stubLess chance of blowing off your whole leg.05:50
lifelesssee -dev for some rambling about this :)05:58
poolieoh, i remember06:00
poolielifeless: the thing i was thinking of was the failure of the ssh server to talk to the internal xmlrpc06:01
pooliexmlrpc was failing to look up a branch (iirc)06:01
lifelessstub: francis is excited by our progress :)06:05
poolieah, it's not a readonly query though06:05
lifelesswhich bug ?06:08
_mup_Bug #294726: "ProtocolError for xmlrpc.lp.internal:8097/branchfilesystem" when pushing a branch <lp-code> <Launchpad itself:Triaged> < https://launchpad.net/bugs/294726 >06:08
pooliei replied06:08
poolieespecially #406:09
poolielifeless: these are interesting cases because of course a person is quite likely to want to link a branch, or push it again, at the time it's being scanned06:14
wgrantOr create an MP...06:15
wgrantThat's where I see it most.06:15
pooliealso these seem like fairly interesting things to put into a message queue or into a non sql database06:15
pooliethough, probably that's not the best short term step06:15
wgrantLaunchpad has very few good short term steps :)06:16
stublifeless: Bug is the other table with similar issues to Branch. Want to move heat and heat_last_updated to a separate table, probably the other timestamps and caches too.06:16
lifeless(I agree, I haven't thought deeply about where or how.06:17
lifelesse.g. should different fields be in separate tables06:17
lifelessor a k:v table we get row locks on06:17
lifelessor ...06:17
pooliei saw your comment reccently about "generally lp doesn't want ...options"06:17
pooliewhich i agree with in many ways06:17
pooliehowever it was in the context of notifications06:17
poolieand i do think a fb or g+ style "do/don't tell me about X action" per user would be ok, don't you?06:18
lifelesshowever, we have a bug asking for reasonably comprehensive subscriptions to teams06:18
lifelesswhich is worth doing directly I think, rather than an awkward one off in a corner06:19
pooliewhat does "subscription to teams" mean?06:19
poolietelling you about changes to memberships?06:19
lifelessand possibly other metadata06:19
=== al-maisan is now known as almaisan-away
lifelessstub: 0.0.2 of pgbouncer is in the download cache for you06:25
lifelessstub: I have to run, but I'd love to know whats left to do a no-op downtime run ?06:26
lifelessstub: I am guessing docs and landing last nights branch ?06:26
stublifeless: We need to push out configs to make everything go via pgbouncer.06:26
stublifeless: Apart from that, we can test now. Last nights branch is an improvement, but not necessary. And docs I prefer to write after I've confirmed everything works rather than having to adjust them for reality later :-)06:27
stubpgbouncer configs is the slow step06:27
lifelessI've blogged about this, as promised06:27
lifelessso users won't be shocked06:28
wgrantDoes this mean we need downtime for all downtime services in the next couple of days?06:28
wgrant(librarian being the main problem)06:28
lifelessah the uploader ? yes we do06:29
stubThink librarian actually reconnects happily - did on staging anyway when I ran this stuff.06:29
wgrantWe restarted it two weeks ago without hiccups, but still, something to consider.06:29
lifelessstub: wgrant is referring to migrating it to use pgbouncer06:30
lifelessstub: uploads to it are not load balanced at the moment06:30
stubAhh, yes. But outages will be minor (test connectivity first, then bounce).06:30
stubIs the upload port going via haproxy?06:31
wgrantNot yet.06:31
wgrantFirewall rules are unclear.06:31
stubit shouldn't be a problem being generous with access to the upload port. Not much you can do apart from try and fill up our disk.06:32
StevenKThere is no try06:32
StevenKThey *can*06:32
lifelessuploads go via appservers06:33
lifelessso an interrupted librarian results in an appserver OOPS06:33
lifelessits not a big deal06:33
wgrantYes, let's pretend that scripts don't exist :)06:34
lifelessthey can be queisced prior to the librarian bounce06:35
lifelessstub: sounds like the config update is the critical path thing then; if you need anything from me on it, let me know, otherwise its in your court06:35
wgrantSo, codehosting, ftpmaster, librarian, mailman are the downtime services I know of.06:36
wgrantBut mailman doesn't have DB access, I suppose.06:36
jtvhi lifeless—almost 9AM here now.  What's up?06:57
rvbaGood morning.07:05
jtvgood morning rvba!07:06
jtvStevenK, wgrant: mind if I update dogfood and run generate-contents-files?07:07
wgrantPlease do.07:07
jtvrvba: dogfood conflict in package cloner.  Were you running an experimental branch there?07:12
henningedanilos: Hi! ;)07:21
jtvmorning, former teammates07:25
=== wgrant changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 238 - 0:[#######=]:256
henningeHi jtv!07:26
henningejtv: What do you think should be done to fix bug 504062?07:26
_mup_Bug #504062: External suggestion "Used in" links to disabled template <404> <lp-translations> <oops> <trivial> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/504062 >07:26
* jtv looks07:27
henningejtv: Should non-admins not see the suggestion at all or just don't get a link to the template?07:27
rvbajtv: sorry, was afk ... I might know what the problem is ...07:28
rvbajtv: how can I take a look? (I don't have access to DF)07:28
jtvrvba: ah.  Hang on, have to look at something else now.07:28
pooliespiv: re your question on bug 813349, people filed a couple of nit type bugs07:28
_mup_Bug #813349: hard to get from mp to per-revision diffs <code-review> <javascript> <qa-ok> <ui> <Launchpad itself:Fix Committed by spiv> < https://launchpad.net/bugs/813349 >07:28
poolielike your link colour is wrong07:28
pooliepersonally i wouldn't block on them07:28
jtvhenninge: istm the reason for showing suggestions from disabled templates is mostly obsolete thanks to sharing.07:29
jtvhenninge: we could try getting some data — how often are those translations both different and useful?07:30
jtvIf the answer is "rarely," then the solution is simple.07:30
jtv(Well, apart perhaps from lag between the time the template is disabled and the next update of the suggestive-templates cache).07:31
jtvrvba, here's the conflict: http://paste.ubuntu.com/652250/07:32
jtvhenninge: if it turns out we're getting real benefit from disabled templates, then we'd have to worry about disabling links.  I even think Zope would do that for us if this were a permission error rather than a 404, but at a performance cost.07:34
adeuringgood morning07:35
henningeHI adeuring!07:35
adeuringhi henninge!07:35
rvbajtv: It's Julian SQL optimisation from yesterday. It looks to me like he tried it manually on DF, then he got it properly landed because (I think) all the code that causes the conflict is there already.07:36
henningejtv: Hm, I'd like to keep this simple ...07:36
jtvrvba: in that case, I should be able to revert & pull.  That's great.07:36
rvbajtv: I think so yes.07:36
jtvhenninge: I sympathize, but if it were truly simple maybe this bug would've been fixed by now.  :)07:37
henningejtv: Before I read this I would have expected translations from disabled templates to not show up07:37
jtvYes, I thought they were excluded from the suggestive-templates cache.  Have you been able to reproduce the problem?07:38
henningejtv: on qastaging, yes. But I am not sure about the result being cached.07:38
henningejtv: how long does stuff live in that cache?07:39
spivpoolie: hmm, it'd be nice to have those bugs linked to 813349 perhaps, but probably not important if they're not blockers07:39
henningealso, I never worked with the cache before.07:39
jtvhenninge: the suggestive templates cache is rewritten in its entirely… I think daily at the moment, but may be much more frequently.  We could afford to do it much more often than that.07:40
spivpoolie: my main concern is that the bug doesn't languish indefinitely as Fix Committed.07:40
jtvhenninge: it's just a database table containing the ids of POTemplates that can be used to get suggestions from.  And it does indeed exclude disabled templates.07:41
jtv(Just checked)07:41
pooliespiv so fortunately 'new bugs' should find them07:41
jtvhenninge: so whenever a template is disabled, this problem can exist until the cache is rewritten.07:41
jtvhenninge: (Also, I see it hasn't been updated for usage enums)07:41
jtvhenninge: The bug description therefore would seem a little out of date.07:42
henningejtv: so the problem is really just about how to deal with an invalid cache.07:43
jtvYes, it's cache invalidation — one of the two things that are hard.07:43
jtvWe could afford to update the cache whenever anyone deactivates a template.07:44
jtvIIRC it takes only a fraction of a second to find all those templates; it's just that we were doing it multiple times for every translations page.07:44
jtvIf the suggestions queries also join in the POTemplate, we could just add a redundant iscurrent check.07:45
pooliespiv if only there was automatic connection between related bugs :)07:46
henningejtv: yes, those were the two things I was considering right now.07:46
jtvAlas, the query doesn't include POTemplate.07:46
henningejtv: I just wasn't sure how expensive the joining would be.07:47
jtvAnd this is velly velly pelformance-snsitive.07:47
jtvSo best not add that join.07:47
henningejtv: but removing that entry from the cache should be simple.07:47
jtvWhat's really weird by the way is that the code in _getExternalTranslationMessages seems to fetch POTMsgSet ids and call them "pots."07:48
pooliespiv, added07:48
jtvMaybe that's just someone who wasn't familiar with the conventional abbreviations.07:48
jtvhenninge: it looks like someone's made a mess of that code.  There's also an over-long line of SQL, with the keywords not capitalized.  That smells of someone just trying something out and accidentally landing it as soon as it worked, without review.07:49
spivpoolie: thanks07:49
jtvhenninge: ah no, it's lifeless code.07:50
henningejtv: how is that different?07:51
jtvThis is hard to say diplomatically.07:52
jtvIn Launchpad we have traditionally focused on producing code that's easy to read.07:52
jtvRobert leans more towards keeping it easy to write.07:52
jtvSo I often struggle reading his code.07:52
jtvFor that reason I don't think it saves any time, but opinion seems divided on the subject.07:53
jtvStill, you would have expected a "make lint."07:53
henningevery much so07:53
* spiv offers hennige a …, for variety07:53
henningethanks spiv ;)07:53
jtvAlso, try <compose>..07:53
spivjtv: how do you think I typed it? :)07:54
jtvspiv: I do apologize.  :)  I only checked Henning's.  :)07:54
spivOr perhaps I should say: ☺07:54
jtvI ♥ <compose>07:54
* henninge learnt something07:55
jtv"You've taken your first step into a larger world…"07:55
henningeI already know those …07:56
* spiv offers jtv <compose><"07:57
jtvOooh!  Thank you!07:57
jtvYou know what's going to happen now, don't you?07:57
spiv¡Por supuesto! ;)07:58
jtvmorning bigjools08:01
jtvbigjools: I'm just messing around on dogfood right now, so beware08:02
bigjoolsjtv: eek :)08:02
jtvAh-HAaah.  No "idle in transaction" no more for generate-contents-files.08:04
jtvDeployments to cocoplum do take downtime, right?08:06
wgrantYes. We'll probably need one in the next day or so for config updates.08:06
wgrantstub: Any idea on the timing of that?08:07
stubwgrant: nup08:07
stubasap, but out of my hands now (apart from nagging and whining)08:07
jtvhloeung: about https://code.launchpad.net/~jtv/lp-production-configs/config-810989/+merge/68867 — shall we just merge & deploy it then?08:11
jtvhi mrevell08:12
jtvwgrant: any idea whether staging and qastaging have the ubuntu-contents dir?08:13
jtvIn other words, whether we ever run generate-contents-files on them (whether from cron or for testing)?08:13
wgrantjtv: They should not.08:13
jtvThen I'll not worry about it.08:14
wgrantAFAIK nobody has ever been foolish enough to run a publisher on them.08:14
jtvhuwshimi: thanks for the icon by the way.  I'll just bung it into l/c/l/images and update the code to use it.08:17
huwshimijtv: Great, no problems.08:17
=== almaisan-away is now known as al-maisan
henningejtv: so, should I put the removal into the admin view code (which AFAIK is the only place where templates can be disabled) or in the model code (i.e. introduce a setter).08:22
henningejtv: I'd prefer model code I think08:23
henningecool, great to see we agree …08:23
jtvAt this point it may possibly be worth reviving the utility.08:24
henningejtv: which utility?08:26
jtvI originally wrote a utility for this cache, but IIRC the reviewer thought it was too much.08:26
jtvUp to you to draw the line of where it deserves its own utility though.08:27
henningejtv: oh yes, I noticed there is no model code for it.08:28
jtvIf you want to add that, just be sure to define the template id as the primary key.08:29
jtvI don't think we really need it though.08:29
henningeI don't either ...08:30
henningequick question08:33
henningegetUtility(IStoreSelector).get(MAIN_STORE, DEFAULT_FLAVOR)08:33
henningeis now equivalent to08:34
=== gmb changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: gmb | Critical bugs: 238 - 0:[#######=]:256
henningeoh, I don't have "something" here ...08:34
jtvhenninge: no, default flavor is IStore(something).  May be a class.08:40
jtvIMasterStore is the master flavor.08:40
henningeI see08:40
jelmergmb: Thanks for the re-review and ec2 submission08:46
lifelessbigjools: you know how I said 10m was ok for a backend script the other day ?09:38
lifelessI was wrong; see my performance-tuesday mail from today :)09:39
* bigjools reaches for razor blades09:39
* bigjools is not sure whether to use them on himself or lifeless09:39
lifelessjtv: I pinged you when I was looking at some locking / concurrency/contention innards of postgresql09:40
lifelessjtv: for a teddy bear, but I sorted itout09:40
wgrantlifeless: Well, it should be OK for some things :)09:42
bigjoolslifeless: I'd say that doing txns in less than 2.5 seconds is lala land09:43
bigjoolsunless the magic performance fairy visits the packagecopier09:43
bigjoolsand somehow we get divine intervention in the soyuz schema and all the code gets rewritten to avoid races and contentions09:44
wgrantSome sorts of transactions are OK.09:45
jtvTeddy bear..?09:45
lifelessjtv: someone to talk about a problem with09:53
lifelessbigjools: so, if we can't get to 2.5 seconds consistently, we can never get to 5 seconds reliably if there are any common things being changed.09:56
lifelessbigjools: its not something we can wave a wand to have happen, but its definitely doable.09:56
bigjoolslifeless: I am OTP but see our conversation this time yesterday09:57
bigjoolsfor all the scary reasons why this is hard09:57
bigjoolstl;dr = consistency issues09:57
lifelessbigjools: sure, so the key question is how we can reduce the risk / find ways to assess it.09:59
bigjoolswe kinda ruled out using staging tables09:59
bigjoolswhich means fixing the rest of the code.  And we're not sure what needs fixing.10:00
lifelessthe first thing that occurs to me is divide and conquer:10:00
lifelessbreak the problem code (whatever it is) into a few categories:10:00
lifeless - readonly code that we can tolerate stupidity in - things with ephemeral output like queuing decisions or web ui10:01
lifeless - readonly code that must be accurate like archive publication and contents generation10:01
lifeless - writing code (which we assume we cannot tolerate stupidity in)10:02
lifelessthen, look for queries that reference distroseries - probably a few hundred tops that do that; toss them into each of those categories as much as possible, and things that are ok we can just move on.10:02
bigjoolsyeah, not saying it's impossible, just a tedious and error-prone job10:03
bigjoolsI'm interested in re-assessing our model to prevent this crap10:04
bigjoolswe need a discussion at the next T'Dome10:04
bigjoolsthe basic issue is that the model doesn't prevent the inconsistency10:05
lifelessthats a pretty core ingredient :)10:06
bigjoolslifeless: yes - there's another timebomb as well, I'll tell you about it in -ops10:07
jtvgmb: review?  https://code.launchpad.net/~jtv/launchpad/bug-800573/+merge/6924010:22
gmbjtv: Sure10:22
gmb'Huw "Emo" Wilkins'? Nice :)10:23
gmbjtv: I've come to expect fairly weighty branches from you. Nothing like breaking stereotypes, eh? Approved.10:26
jtvgmb: a lot of that is just plain bad luck, I'm afraid.10:27
jtvNot all.10:27
jtvMaybe you're just my reviewer of last resort, the one I end up with when everyone else has turned me down.10:27
=== al-maisan is now known as almaisan-away
jtvgmb: thanks by the way.  :)10:39
gmbnp :)10:40
jtvbigjools, cjwatson: is the -commercial pocket completely and utterly gone now that dapper is eol?  Can we drop the publisher provisions (the commercial-compat.sh script)?10:46
jtvwgrant also said to talk to the releases team, IIRC — if so, what's the right communication channel for that?10:47
cjwatsonyes, I think it is10:49
* gmb lunches10:50
wgrantDapper is officially EOL, but someone may be playing OEM-like tricks with it...10:50
bigjoolsjtv: talk to skaet10:50
henningejtv: any idea why I might get this? ForbiddenAttribute: ('__call__', <lp.testing.fakemethod.FakeMethod object at 0xdc8d74c>)10:51
cjwatsonjtv: you should also check with Brian Thomason (iamfuzz), since he does 99% of the work on partner10:54
cjwatsonif he's no longer uploading anything to dapper-commercial, then it can die10:54
cjwatsonI expect he'll in fact be more directly up to date on this than skaet will10:55
cjwatsonbigjools: you asked me to prepare that staging PPA with changes intended for the Launchpad PPA (https://lists.launchpad.net/launchpad-dev/msg07686.html) - are you also able to review it?10:56
bigjoolscjwatson: someone on the maintenance cycle should review it, I'm kinda busy.  Perhaps gmb?10:56
jtvhenninge: fakemethod behind a proxy, I think.11:03
jtvbigjools: thanks11:03
henningejtv: yes11:03
henningejtv: I was patching a utility. bad idea11:03
jtvcjwatson: thanks11:04
jtvhenninge: I guess you're probably already cleaning this up, but just so we don't forget: bug 81636611:23
_mup_Bug #816366: Misleading code in POTMsgSet <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/816366 >11:23
=== almaisan-away is now known as al-maisan
henningejtv: http://paste.ubuntu.com/652366/11:26
jtvhenninge: great, thanks.  Phe!11:27
=== henninge is now known as henninge-lunch
lifelessnight y'all11:38
gmbsinzui: Can you take a look at, and perhaps give a cogent response to https://answers.launchpad.net/launchpad/+question/165200?12:05
gmbhenninge, danilos, jtv: Can one of you good fellers take a look at https://answers.launchpad.net/launchpad/+question/165699 please?12:10
danilosgmb, on it12:13
danilosgmb, btw, can you take a look at https://code.launchpad.net/~danilo/launchpad/bug-800123/+merge/69264 when you find some time?12:21
danilosgmb, thanks12:30
gmbdanilos: r=me12:55
danilosgmb, ta12:55
deryckadeuring, henninge-lunch -- sorry I'm late, coming to standup in a couple minutes.13:32
jtvgmb: still reviewing?  If so → https://code.launchpad.net/~jtv/launchpad/bug-814141/+merge/6928513:55
gmbjtv: Sure.13:56
deryckbac, hi there.14:39
bachi deryck14:39
deryckbac, I was going to do a nodowntime deploy and if the qa page is correct, your fix for bug 788685 is blocking other qa-ok revs....14:39
_mup_Bug #788685: Enable translating selected Ubuntu universe packages in Launchpad <escalated> <oem-services> <qa-needstesting> <Launchpad itself:Fix Committed by bac> <OEM Priority Project:Invalid> <Ubuntu Translations:Triaged> <pkgbinarymangler (Ubuntu):Fix Released by pitti> < https://launchpad.net/bugs/788685 >14:39
deryckbac, any chance of qa'ing that today?14:39
bacderyck: i'll try.  it isn't straightforward14:40
deryckbac, I assumed it wasn't.  do you need a former translations persons help?14:41
bacderyck: luckily i have one.  :)14:41
deryckbac, right, I knew danilos was on your team. :-)  I didn't know if he was still around to help. :)14:42
deryckwas going to offer my translation guy :)14:42
bacderyck: i may take you up.  will have a go now.14:43
bacbigjools: ping14:43
deryckbac, ok, cool.  Thanks!14:43
bigjoolsbac: hello14:43
bacbigjools: i need to QA bug 788685.  danilos suggested yesterday we do it on dogfood and "ask soyuz people about re-uploading (re-publishing or something) existing ubuntu packages on dogfood"14:44
_mup_Bug #788685: Enable translating selected Ubuntu universe packages in Launchpad <escalated> <oem-services> <qa-needstesting> <Launchpad itself:Fix Committed by bac> <OEM Priority Project:Invalid> <Ubuntu Translations:Triaged> <pkgbinarymangler (Ubuntu):Fix Released by pitti> < https://launchpad.net/bugs/788685 >14:44
bacbigjools: would you have a moment or two and does that make sense to you?14:45
bigjoolsbac: you can upload to DF and I can run the upload processor14:45
bigjoolsyou just need to "apt-get source" on an existing universe package and bump its version14:46
bigjoolsbuild the package then dput to DF14:46
bacbigjools: do i just ask a losa to deploy my branch onto DF?14:46
bigjoolsbac: no you ask me :)14:46
bigjoolsis it landed?14:47
bacbigjools: excellent14:47
bacbigjools: yeah, it is14:47
bacbigjools: r1351014:47
bigjoolsbac: is it on db-devel (we run DF off db-devel)14:47
bigjoolsif not I'll merge the branch14:47
bigjoolsdonde esta?14:48
bacbigjools:  lp:~bac/launchpad/bug-78868514:49
bigjoolsbac: ah it was already on db-devel it seems.14:50
bacoh righty14:50
bigjoolsbac: so go ahead and upload, ping me when it needs processing14:50
bacbigjools: about that...14:50
bacbigjools: i need to get a source that has translations, right?14:50
bigjoolsbac: and you need the XS-Ubuntu-Use-Langpack: yes header in debian/control14:52
bacyes, of course i do.  i was just thinking that14:52
bigjoolsand also we need to check that the DF buildds chroots have got the new pkgbinarymangler14:52
=== jtv is now known as jtv-afk
sinzuimatsubara, mumble or skype?15:00
matsubarasinzui, mumble15:00
=== henninge-lunch is now known as henninge
henningegmb: Hi!15:06
gmbHi henninge15:06
henningegmb: Can you please do a review for me?15:06
gmbhenninge: Approved15:23
henningegmb: thanks15:23
=== gmb changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 238 - 0:[#######=]:256
=== al-maisan is now known as almaisan-away
henningegary_poster: Hi!15:52
gary_posterhey henninge :-) what's up?15:52
henningegary_poster: I am looking at bug 48151215:53
_mup_Bug #481512: race condition when rotating logs <canonical-losa-lp> <escalated> <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/481512 >15:53
gary_posterhenninge, cool15:53
henningegary_poster: but I am not familiar with the logging system in LP.15:54
henningegary_poster: I hoped that you could give me any input you have on that bug.15:54
gary_posterhenninge, we send a signal which is supposed to rotate the logs.  I found some pertinent code recently; lemme see if I can find it.15:54
gary_posterhenninge, still looking, but lib/canonical/launchpad/webapp/sigusr2.py15:56
* henninge looks15:56
henningeok, basic stuff15:58
henningethere is a test case for it15:58
=== salgado is now known as salgado-lunch
=== deryck is now known as deryck[lunch]
gary_posterhenninge, I've looked for more--I thought I saw some custom log handlers but the only ones I can find now are for tests and for scripts (and a simple one that adds custom log levels).  So, I'm afraid ZConfig might be the place to look for that.16:02
gary_posterAuthor of that log rotate code in ZConfig is Fred Drake, but I don't think knowing that helps us any :-)16:02
gary_posterso that's it henninge16:02
henningegary_poster: thank, that's a start16:03
gary_postercool henninge, thanks for looking at that one16:03
henningegary_poster: it was escalated, so somebody has to do it ... ;-)16:04
bacderyck[lunch]: making progress16:46
=== matsubara is now known as matsubara-lunch
bigjoolsgood night all16:57
=== deryck[lunch] is now known as deryck
=== salgado-lunch is now known as salgado
=== henninge is now known as henninge-afk
=== beuno is now known as beuno-lunch
=== matsubara-lunch is now known as matsubara
=== beuno-lunch is now known as beuno
sinzuijcsackett, do you have time to mumble20:16
jcsackettsinzui: sure.20:17
jcsackettsinzui: i'm on mumble now.20:18
LPCIBotProject devel build #920: FAILURE in 5 hr 39 min: https://lpci.wedontsleep.org/job/devel/920/20:44
lifelesssinzui: hi21:07
=== matsubara is now known as matsubara-afk
deryckLater everyone.21:17
sinzuihi lifeless21:27
lifelesssinzui: hi; I've forwarded you a mailling list test failure21:28
sinzuiI saw21:28
lifelessdo we use the --notifications option ?21:28
lifelessand do we want it to override this quieter behaviour ?21:28
sinzuiThe list import is an interesting case. I think we can drop it though since21:28
sinzuilifeless, I really do not know about --notifications21:29
sinzuiI was not certain what was going on in the archive email failure. Do we add users to teams when they are subscribed to archives?21:30
lifelesssinzui: the test made an open team21:30
sinzuiI really do understand that case then21:30
lifelesssinzui: so it then purged the mails that adding someone to the team generated. (factory.makeTeam makes OPEN teams)21:30
lifelesssinzui: e.g. the archive failure was trivial and you can ignore it.21:31
lifelesssinzui: the comment above the failing line said (paraphrased) 'ignore the mails about joining the team'21:31
lifelesssinzui: so, I'm trying to change the test_mlists test to use a moderated team21:33
allenapNight all.21:36
sinzuilifeless, we have run The mailing list import script twice21:37
sinzuilifeless, it is run by a losa to migrate a list. it create and find users from the email address in the foreign list id21:38
sinzuilifeless, I think the emails are spam in this case. When the import is done, the team admin is going to inspect the  team subscription page and ask for the log of the import21:38
lifelesssinzui: this fixes the test: http://paste.ubuntu.com/652682/21:39
lifelesssinzui: but if you want, I could rip out the --notifications feature entirely.21:39
sinzuilifeless, keep the test chnge21:40
sinzuiI do not think this script will be used until it can be used via the UI21:40
lifelessok, so that pastebin is ok with you ?21:41
sinzuiyep. +121:41
lifelessI will land this change then.21:41
sinzuiStevenK, mumble?23:01
sinzuiStevenK, http://pastebin.ubuntu.com/652721/23:05
=== StevenK changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: StevenK | Critical bugs: 238 - 0:[#######=]:256
=== wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: StevenK | Critical bugs: 240 - 0:[#######=]:256
wgrantThese "Job ran too long" scanner OOPSes really suck.23:50
wgrantsinzui: https://code.launchpad.net/~wgrant/launchpad/transitionToTarget-series-sourcepackage, if you have time.23:53

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!