[00:38] Weren't the "technical difficulties" resolved yesterday? [00:38] if you mean a somewhat critical server that haemorraged? yeah. whats up? [00:39] It's still in the #launchpad topic. [00:39] haha. thanks! :-D [00:43] 'technical difficulties' is such a useful term [00:43] Yep. [00:44] it covers everything from things going a bit slow to the data centre burning down [00:46] Or just SKS being crap, in this case.. [00:46] which managed to hold up the appserver? [00:46] Yes, all of them, I believe. [00:46] ouch [01:00] it wasn't sks' fault [01:00] (this time) [01:46] How long is it supposed to take for Launchpad to notice a new source upload to Debian? [01:47] It is meant to look twice a day, or maybe four times if the sysadmins have actioned that RT. [01:47] Note that some appeared to be failing a couple of weeks ago, and slightly over 1% of the archive is currently unimportable because of the format transition. [01:49] It's times like these when it would be handy for the production crontabs to be opensource [03:08] mwhudson: probably not best to reply to Bug 236973 with "oh crap!" [03:08] Bug #236973: the interval between launching a new code import job should be more flexible [03:11] thumper: no, probably not [03:11] * spm subscribes losas to that one [03:11] mwhudson: fwiw, that "cheat" in the crontab is evil; so just getting rid of that would be wonderful. [03:12] spm: so I guess you wouldn't be super happy for me to suggest fudging it harder? [03:12] meh [03:12] :-) [03:13] thumper: the fix proposed in the description would be very simple and certainly help a bunch [03:13] mwhudson: srsly tho - fudge in what way? we could do some things that wouldn't be quite so ... yuk, but still be a fudge. [03:14] spm: well, s/sleep 30/sleep 20/ and similar [03:14] spm: probably better to do it in the script and avoiding paying the startup cost [03:14] 'k. a proper wrapper script vs evil. [03:14] right [03:15] mwhudson: have we discovered/can-find the actual startup time of the python code to make stuff happen? that'd assist in giving a lower bound. [03:15] spm: it's probably 1.something seconds in this case [03:15] oki [03:15] spm: you could look for the times of the requests to the CodeImportScheduler endpoint on arsenic [03:16] that's the first actual useful thing the script does [03:16] as an offset from 0 or 30? yeah! good idea! [03:16] so if requests come in at xx:xx:05, it's a 5 second startup [03:16] nods [03:20] mwhudson: eyeball suggests ~ 3-5 secs off :00 and ~ 36-38 off 30 - but I'd suggest the latter is simply due to the effect of the 1st startup etc [03:20] spm: right, that makes sense [03:21] * spm ponders if breaking out R and Doing It Right is worth it.... ;-) [03:25] mwhudson: thinking aloud here - so shoot down if needed - but I wonder if having a random offset sleep wuold be better. eg a random time between 5-15 seconds - would go some small way to avoid the thundering herd of 3 machines all doing the same stuff at the same moment. [03:25] spm: yes, that's certainly possible [03:25] and being random, would rely on us losas having to sync eg 15 machines down the track with subtle offsets [03:25] *wound't* [03:26] wouldn't even bleh [03:26] * wgrant wonders how they could place so much load on anything in order to make that necessary. [03:27] mainly as evetually you will; better to design in the workaround up front [03:27] peak loads always bite you hard when you least want them too :-/ [03:29] esp as these sort of things tend to not impact linearly - more like exponential; (not) literally the straw breaks the camels back [03:31] I think it was cnet? one of the big news orgs. in the earlier days of RSS, they used to get *hammered* at specific times. all due to gazillions of teeny trivial requests for rss feeds from clients using the same s/w that was hard wirded to timee X; vs time X + random [04:08] so, bug tags don't apply across projects? I can't click the "yui-3-upgrade" tag to see all problems across all of LP?? [04:12] ah, to see all of the bugs with a particular tag across projects, I need to search for that tag on the super-project. [04:24] jtv, ping [04:25] mars: yes? [04:26] hi jtv, just wondering, I need someone to help test the translations JavaScript upgrade on staging. Would you be willing to do so? [04:26] or perhaps volunteer a teammate? :) [04:29] I can help, but hang on [04:29] got a bit of a situation involving contact lenses :) [04:29] jtv: you pinged last night? I gather all is solved? or ... ? [04:29] :) [04:30] spm: yes, thanks [04:30] sweet [04:33] mars: I know one of ours broke with the upgrade (something I wrote in fact), but last night db-devel already had that code changed—by henning I guess [04:34] ahhh! staging's had a code update [04:34] mars: so... what is it you want to test? [04:34] jtv: gawd I hope so - it's been runing for long enough..... [04:34] spm: a code update? I thought those were very lightweight? [04:35] was a full one. DB ++ [04:35] ah well that takes a while yes :) [04:36] jtv, everything in translations that might not be covered by windmill. [04:37] jtv, if you do find anything, could you please tag it as "yui-3-upgrade"? That way we can see what blocker we have for the next rollout. [04:37] mars: ok... we don't have much, so it shouldn't take long [04:38] jtv, cool, thank you for the help! [04:38] np [04:39] mars: oh, and there's something unrelated I've been wondering about [04:39] the spinner [04:39] I can see why it's not a sprite, [04:39] but that does mean it can take a while to load [04:40] so what I did on the import queue is put a spinner at the bottom of the page while the JS is setting things up. [04:40] It really helps, but... have I been re-inventing the wheel? [04:40] I should be clearer: [04:41] jtv, soyuz had the same problem. Each page has to decide when to load that image so that it is available. [04:41] I made the JS load the spinner early on (by showing it at the bottom of the page) so that when it's actually needed, it's in the browser cache [04:41] At some point, we hit the lower limit of our ability to serve files over-the-wire. [04:41] mars: well, that limit can be lowered :) [04:41] the lower limit is zero... not sure I follow [04:42] lifeless: GMTA [04:42] jtv, once someone caches the spinner, it should be available right away, no? [04:43] are you only worried about initial page loads? [04:44] I find the spinner needs to be re-fetched fairly often === mzz_ is now known as mzz [04:52] spm: there is something _else_ you could help me with... [04:52] jtv: sure [04:52] spm: I need Q/A on a bit of UI that's currently only available to admins [04:52] spm: e.g. here: https://translations.staging.launchpad.net/ubuntu/+source/apport/+custom-language-codes [04:53] jtv: so you want rubber ducky to give it a whirl? [04:53] do you see a list there that you can add items to and remove items from? [04:53] I am not familiar with this rubber ducky you refer to [04:53] jtv: the rubber duck is the icon of LP admins [04:54] https://edge.launchpad.net/~admins [04:54] * thumper EODs [04:54] thumper: g'night [04:54] jtv: so that came up with "No custom language codes have been defined." and "Add a custom language code " with a green plus [04:54] night thumper [04:55] selecting the + then gave a list of langs to choose from [04:55] spm: good so far [04:55] spm: maybe I should just join that team... it's got open membership anyway [04:56] jtv: whatever you think is best? it's only for a short period anyway :-) [04:56] spm: I was lying for comic effect... it's restricted. :) [04:56] * spm is scored upon. again. :-) [04:57] hardly! [04:57] you didn't throw a tantrum [04:57] (unfortunately, but that's another matter) [04:57] anyway, can you now click into the custom language code you just created to view it? [04:59] jtv: https://translations.staging.launchpad.net/ubuntu/+source/apport/+customcode/spm spm-is-ace (hahahahah) [04:59] mars: I was wondering if maybe we should have some easy-to-use boilerplate for "go fetch my spinner in case I need it later." To be done at the very end of loading everything else. [05:00] jtv: "For apport in ubuntu, uploads with the language code “spm” are associated with the language Acehnese" [05:00] spm: yup, works for me too. [05:00] sweet [05:00] jtv, I just tried loading the bugs page on edge, and the spinner was cached by the browser. Of course, that doesn't mean something something else could not be blocking it from rendering instead [05:00] spm: and as is appropriate, I don't see the "remove" link but you probably do [05:00] jtv: sure do. give it a twirl? [05:01] spm: oh yes... the red icon shown in the listing, please! [05:01] jtv, we thought about that for soyuz, but we did not have a second case for a while. Bugs also has a spinner, but I don't know if they optimized it as well. [05:01] mars: some pages you end up reloading a lot [05:01] jtv: "Removed custom language code 'spm'." == gone [05:01] jtv, reloading the page contents, or the spinner graphic? [05:01] mars: the page contents, but my impression was that that made the spinner appear slowly as well [05:01] spm: cool! [05:02] spm: now, the same thing should appear also in a different kind of context: [05:02] https://translations.staging.launchpad.net/expono/+custom-language-codes [05:02] spm: that's a project, whereas the previous one was a package [05:02] kk, go thru the same process? [05:02] jtv, it could. Depends on what is in the page. For instance, putting