[02:02] Ah, the Soyuz OOPS report is a bit happier today. [02:27] lifeless: :( [02:27] Archive:+index's soft timeouts have gone away. [02:28] Our exception counts are 20% lower than pre-release, soft timeouts down by more than 60%. [02:28] Yay. [02:45] wgrant: cool [02:46] 219 / 432 BugTask:+index [02:46] 90 / 241 Distribution:+bugs [02:46] next up [02:46] I think Monday is 'remove substring search' day [02:56] Heh. [02:58] Hmm [02:58] Unbreak checkwatches, or split it into its own report? [02:58] The former will take quite a while, I suspect. [02:58] Particularly since it uses OOPSes somewhat differently to the rest of the world. [03:03] wgrant: unbreak [03:03] splitting it into its own report makes the daily report analysis harder, while not making the checkwatches component easier. [03:03] its a pessimism - just like having a soyuz specific report [03:05] Right. [03:05] But checkwatches is designed to use OOPSes for warnings. [03:06] we should get an oops when we need to take an action [03:06] checkwatches seems to be in a little bit of a grey area [05:05] Hmm. [05:05] We seem to have a major performance issue with update-bugtask-targetnamecaches. [05:05] It is again taking >18 hours. [05:06] I think I will delete it. [05:07] :( things still use it. [05:07] Perhaps selecting distinct targets and updating changed ones will be better and easy. [05:11] Well, that looks like it should take about 10 seconds. [05:12] Yeah. [06:20] Project devel build (436): FAILURE in 5 hr 37 min: https://hudson.wedontsleep.org/job/devel/436/ [08:51] wgrant: what uses it? [09:00] lifeless: I guess search is the only thing that can't calculate it on-the-fly. [09:00] But tables and titles use it too. [09:03] grah [09:03] well, fix that. [09:03] no good reason for it to be used; we load the related objects always anyway. [09:04] Fix what? The non-search uses? [09:04] yes [09:05] the search use I'm going to put behind a FF tomorrow [09:05] Aha. [09:05] Then turn it off and see if anyone notices? [09:05] hey, why does the cache updater touch every bug? surely just 'bugs changed since the last run' is sufficient. [09:05] product renames, I suspect. [09:06] wgrant: well, turn it off for lp devs, test the improvement, then wider etc. [09:06] :( [09:06] wgrant: products have a datestamp [09:06] A 'when my display name was last changed' datestamp? [09:06] wgrant: doesn't matter [09:07] actually no [09:07] product doesn't have a mod date [09:07] *fail* [09:07] That's what I thought. [09:07] Most stuff doesn't. [09:07] The only mandatory fields are owner and datecreated. [09:07] ok, so product and sourcepackagename [blech - this is crazy to normalise such fields] - need a last_modified [09:08] and distribution [09:08] or [09:08] just ditch all use of it [09:08] its really uninteresting [09:08] I think we can remove all uses by friday, with care and caution [09:08] Yeah. [09:09] We could disable it until then. [09:09] new data is updated by hand, right ? [09:09] I mean, new bugs have a valid cache value [09:09] Yes. [09:09] I'm not sure if task moves are. [09:09] But I presume so. [09:10] woo 12365 hath landed [09:10] Yep. [09:10] hmm [09:10] Does staging have the render time on it yet [09:10] It should... [09:11] Even the FF should have propogated by now. [09:11] nope [09:11] https://staging.launchpad.net/+feature-rules [09:16] wgrant: loggerhead really needs some time [09:17] wgrant: jam has prepped patches to fix the biggest issues; they need review and landing [09:17] Ah, great. I'll have a look tomorrow. [09:17] that would be awesome === almaisan-away is now known as al-maisan === al-maisan is now known as almaisan-away [15:43] Project db-devel build (362): FAILURE in 5 hr 18 min: https://hudson.wedontsleep.org/job/db-devel/362/ [15:43] Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 12367 [15:43] included. [16:01] Project devel build (437): STILL FAILING in 14 min: https://hudson.wedontsleep.org/job/devel/437/ [20:15] morning [20:15] tethering with Rachel's phone is working [20:16] so I'm off to town to get a haircut :) [20:17] mwhudson: what is 'mkdir -p /var/tmp/vostok-archive [20:17] ' [20:17] for [20:20] lifeless: i think it's a DocumentRoot for apache [20:20] that stuff is all stillborn though :/ [20:21] mwhudson: also https://code.launchpad.net/~mwhudson/lp-production-configs/no-more-launchpad-loggerhead/+merge/24366 [20:21] mwhudson: land or delete IMO [20:22] oh oops, that should be landed [20:23] lifeless: do we still use the production-devel and production-stable branches? [20:24] no [20:25] we deploy from stable [20:25] lifeless: so the config-manager/production-{stable,devel} files can go? [20:25] we have a vestigial process to use production-stable if we have something that has to be embargoed all the way [20:25] we haven't used it in 6 months though [20:25] i guess i don't need to care about those files referencing launchpad-loggerhead, anyway [20:26] mwhudson: also, while you are around [20:26] whats the deal with the different theme in lp's loggerhead [20:26] at the time the idea was just to make the launchpad loggerhead blend in with the lp colour scheme a bit more [20:27] mwhudson: it seems to be a different colour scheme than lp itself has though ? [20:27] it may have been less different once [20:28] the 'codehosting orange' matches up, at least [20:28] codehosting oragne? [20:28] * mwhudson realizes that that probably wasn't useulf [20:29] lifeless: the colour the h1 is in on https://code.launchpad.net/~mwhudson/lp-production-configs/no-more-launchpad-loggerhead [20:29] is used in the lp loggerhead theme [20:29] oh [20:29] I'd always thought the h1's were arbitrary [20:43] nice [20:44] best overall 99th percentile yet, I think [20:44] 2.63 - https://devpad.canonical.com/~lpqateam/ppr/lpnet/latest-daily-categories.html [20:44] 6M renders [21:00] Morning. How can I register my application with login.lp.net so that I can get group membership information? [21:00] * mwhudson listens carefully [21:12] losas do that [21:42] lifeless: sorry, I don't understand [21:50] michaelh1: you need to talk to a losa [21:50] the operators of login.ubuntu.com [21:50] michaelh1: login.lp.net is just a theme on l.u.c. [21:52] * thumper is futzing around with lazr.restful again [21:52] tests failed on Friday [21:52] due to the fix I did [21:52] the tests need to be updated, as it is working now as designed [22:32] wgrant: around? [22:33] lifeless: Yup. [22:33] mwhudson: Except that codehosting orange's reign ended yesterday. [22:33] wgrant: I'm wondering if I could ask you to do the rollback for https://bugs.launchpad.net/launchpad/+bug/632847 [22:33] <_mup_> Bug #632847: Bug page OOPS when viewed in deactivated project context <404>

< https://launchpad.net/bugs/632847 > [22:34] wgrant: hence 'at the time' :) [22:34] What's broken? [22:35] wgrant: correlated subquery on every bug search by a logged in user ? [22:35] wgrant: back to product for disabled products - something like 5% selectivity. [22:35] wgrant: its madness and will break performance [22:35] Ah, I hadn't actually looked at the diff yet. [22:36] Have you tested the performance hit on qastaging? [22:36] Yes, it looks bad, but I'd like to have something more concrete before I roll something back. [22:41] ok [22:41] I'm going to make you an admin of ~registry on qastaging [22:42] and now I've removed ~launchpad from it [22:44] OOPS-1870QS55) [22:45] wgrant: I've added another comment [22:45] Thanks. [22:46] mwhudson: What do you think about https://code.launchpad.net/~mwhudson/lp-production-configs/remove-production-max_workers_per_machine/+merge/19671? [22:55] * thumper wipes brow [22:55] phew [22:55] icky tests fixed [22:56] lifeless: Rolling it back. [22:56] But it'll be a couple of hours. [22:56] Since the queue is up to three items. [22:58] wgrant: thanks [22:58] hmm [22:59] I can't seem to login to qastaging with a slightly old lp lib [22:59] launchpad.Launchpad.login_with('hello-world', 'https://qastaging.launchpad.net/') [22:59] -> 404 [22:59] That's no service root. [22:59] api. [23:03] thanks [23:17] lifeless: 86 queries/external actions issued in 1.78 seconds [23:17] lifeless: this is just for devs yeah? [23:17] on the main page [23:17] yes [23:18] its controlled by a feature flag [23:18] thumper: do you like it? [23:18] yep [23:21] morning people [23:22] Ursinha: Hello [23:22] :) [23:23] huwshimi, why aren't you on our team's channel? :) [23:23] Ursinha: Oh [23:23] I understand, we all live in the past :P [23:23] you have a team channel? [23:24] Ursinha: yeah no-one is ever awake there :) [23:24] lifeless, yes sir [23:26] lifeless: Hum, targetnamecache is used for sorting too. [23:26] I wonder if anyone ever uses that. [23:45] lifeless: Do you have a problem with removing targetnamecache from the UI, turning off the cron job (or maybe taking it weekly or something), and hoping that nobody notices that sort order doesn't update when they rename their products? [23:45] when is it used for sorting [23:46] When you order bug search results by location. [23:47] is that used? [23:47] Occasionally by Ubuntu. [23:47] Where targetnamecache never changes. [23:48] It could also be used in a project group context, I suppose. [23:48] sure it can [23:48] retarget source package [23:48] but you mean 'where the thing targeted changes value [23:48] Well, yes. [23:48] wgrant: so, I suggest a similar discussion to mine about search [23:49] blog + launchpad-users + microblog links [23:49] and/or [23:49] web log analysis [23:49] wgrant: we can efficiently support this, if its useful [23:50] Right. [23:50] if its not useful we should drop the column [23:50] wgrant: to answer your question [23:50] a) happy to switch from the targetname cache in the UI to the raw data; that or actually transition to not loading the related things at all, but that seems improbable to me [23:51] b) fixing the cron job is pretty straight forward, if tedious [23:53] wgrant: I agree we need to fix the short term pain [23:53] wgrant: its up to you to choose how, I think. [23:54] I will remove it from the UI and rewrite the script to do a batch update. Requires no model changes, should be a lot quicker, and won't take long to do.