[01:25] i'm getting a failure like [01:25] keyring.tests.test_core: module references __file__ [01:25] Got keyring 0.5.1. [01:25] Getting distribution for 'lazr-js==1.5DEV-r191'. [01:26] i guess this is some kind of dependency out of date thing [01:26] any clues? running update-sourcecode just once did not seem to fix it [01:31] also lazr.restful... [01:55] np, got it [02:56] \o/ [02:56] * 201 Time Outs [02:57] still a long tail === fjlacoste is now known as flacoste [03:06] lifeless: But 43 of those are already fixed. [03:06] 450 soft too, wow. [03:06] We could almost get away with dropping to 9s right now... [03:26] Surprise surprise, all but one or two of the BugTask:+index OOPSes were from gandwana/potassium. [03:54] Project windmill build #180: STILL FAILING in 1 hr 1 min: https://lpci.wedontsleep.org/job/windmill/180/ === almaisan-away is now known as al-maisan [07:15] Oh no, not again. [07:38] wgrant: windmill or something else ? [07:38] wgrant: and yes, the better hardware is making quite a difference [07:39] lifeless: structsub conflict. [07:40] \o. [07:40] Submitting now.... [07:42] coolio [07:42] I think the bug aggregates will need rabbit [07:42] Oh? [07:43] unless I get some metric together on the volume of aggregate-field changes in ubuntu bugs [07:44] Hmmm. [07:44] I may be jumping at ghosts === al-maisan is now known as almaisan-away === almaisan-away is now known as al-maisan [07:44] bbiab === al-maisan is now known as almaisan-away [08:15] wgrant: Structul subscriptions *again*? :-( [08:16] lifeless: I wonder if we can drop the timeout now due to single-threadness plus new appserver of awesome? [08:16] searchTasks is an issue :( [08:17] Apart from searchTasks we can drop 2s (to 9s) without a problem, AFAICT. [08:20] Yippie, build fixed! [08:20] Project devel build #642: FIXED in 5 hr 38 min: https://lpci.wedontsleep.org/job/devel/642/ [08:27] indeed [08:27] we need to address searchtasks - perhaps by result size estimation [08:28] or possibly an exception [08:28] I don't think we can immediately make any vast improvements there :/ [08:30] better search schema [08:30] "immediately" [08:34] fsvo [08:43] tag search is being a pain [09:13] Project windmill build #181: STILL FAILING in 1 hr 0 min: https://lpci.wedontsleep.org/job/windmill/181/ [12:58] Project windmill build #182: STILL FAILING in 1 hr 1 min: https://lpci.wedontsleep.org/job/windmill/182/ [13:34] Project devel build #643: FAILURE in 5 hr 14 min: https://lpci.wedontsleep.org/job/devel/643/ === almaisan-away is now known as al-maisan === al-maisan is now known as almaisan-away [21:14] meep [21:14] 'bug' gets written to 30K times a day [21:14] for ubuntu alone [21:14] 60K across the board [21:18] BugTask:+index is our 10th most popular codepath : 160K hits a day [21:26] Yippie, build fixed! [21:26] Project devel build #644: FIXED in 5 hr 30 min: https://lpci.wedontsleep.org/job/devel/644/ [21:27] Project windmill build #183: STILL FAILING in 1 hr 0 min: https://lpci.wedontsleep.org/job/windmill/183/