| === Ursinha is now known as Ursinha-afk | ||
| === jamesh_ is now known as jamesh | ||
| lifeless | mwhudson: ping; how do I tell if a given loggerhead rev is is production? | 04:59 |
|---|---|---|
| lifeless | thumper: https://bugs.edge.launchpad.net/loggerhead/+bug/518689 - does that need to be private? Seems like no to me. Web crawlers and users can hit that easily enough | 05:01 |
| mwhudson | lifeless: production only lags lp:~launchpad-pqm/loggerhead/devel by at most 24 hours | 05:01 |
| lifeless | and if someone were to dos it we can filter | 05:01 |
| lifeless | mwhudson: its auto rolled out ? | 05:01 |
| mwhudson | lifeless: if you need more precision than that, ask someone who can log into guava i guess (that includes me) | 05:01 |
| lifeless | mwhudson: oh, I see, I need to do missing between the two | 05:01 |
| mwhudson | lifeless: yes | 05:01 |
| lifeless | mwhudson: is lp-pqm's branch a series ? | 05:02 |
| mwhudson | lifeless: no, maybe it should be though | 05:02 |
| thumper | lifeless: Logging as a sec vuln as this is a rather easy way to currently DOS codebrowse. | 05:02 |
| thumper | lifeless: that was from the bug comment | 05:02 |
| lifeless | yeah | 05:02 |
| lifeless | I'm saying I disagree | 05:02 |
| lifeless | using loggerhead is an easy way to DOS is | 05:02 |
| lifeless | s/is/it | 05:02 |
| mwhudson | wasn't that the pygments problem? | 05:02 |
| thumper | lifeless: ok, fair enough | 05:02 |
| lifeless | I'm trying to decide between | 05:03 |
| lifeless | a) public or b) subscribe max | 05:03 |
| thumper | mwhudson: I think it may have been | 05:03 |
| lifeless | mwhudson: yes, but root cause suggests thread pool limits are to blame too | 05:03 |
| thumper | mwhudson: do we have a max pygments limit yet? | 05:03 |
| lifeless | huh | 05:04 |
| lifeless | I can't reject https://code.edge.launchpad.net/~loggerhead-team/loggerhead/trunk-rich/+merge/26010 | 05:05 |
| mwhudson | lifeless: man, the thread pool stuff is so messed up, please fix that | 05:05 |
| lifeless | which is silly, as I created it. | 05:05 |
| mwhudson | thumper: yes, i think so | 05:05 |
| lifeless | is there a bug open about the permissions there? | 05:05 |
| thumper | lifeless: it is a known issue | 05:05 |
| lifeless | shall I file a bug ? | 05:05 |
| thumper | lifeless: and somewhat releated to the proposal status | 05:05 |
| thumper | and that we don't have a withdrawn | 05:06 |
| lifeless | yes, I can see the place in the code | 05:06 |
| thumper | why not delete it? | 05:06 |
| thumper | does it have any real value? | 05:06 |
| lifeless | well, I wanted to be able to point ppl to it :P thats ok, deleted it | 05:06 |
| thumper | lifeless: lets subscribe max for now | 05:06 |
| thumper | what is his lp id? | 05:06 |
| mwhudson | mkanat i think | 05:07 |
| thumper | subscribed | 05:08 |
| lifeless | thumper: already openned it based on your ok, fair enough able | 05:09 |
| lifeless | *above* | 05:09 |
| lifeless | mwhudson: the paste guts seem clear enough, it knows how many total threads, it should be able to have a hard cap. | 05:13 |
| mwhudson | lifeless: but it has this strange concept of 'hung' threads and if threads are 'hung' it spawns more | 05:14 |
| lifeless | yes | 05:14 |
| lifeless | the specific check is | 05:14 |
| lifeless | spawn_if_under | 05:15 |
| lifeless | + hung threads don't count as workers | 05:15 |
| mwhudson | lifeless: probably the limits are way too generous, we should kill threads more aggressively | 05:15 |
| lifeless | that too, but killing threads is likely to show up locking bugs - mutex ownership issues, etc. | 05:15 |
| lifeless | losa | 05:15 |
| lifeless | how many requests/sec does loggerhead suffer? | 05:16 |
| mwhudson | the limits are set in scripts/start-loggerhead.py | 05:16 |
| mwhudson | in the launchpad tree | 05:16 |
| lifeless | grah | 05:16 |
| mwhudson | lifeless: https://lpstats.canonical.com/graphs/CodebrowseHTTPResponses/ | 05:16 |
| lifeless | thats backend? | 05:16 |
| lifeless | man | 05:17 |
| lifeless | I was going to book plane tickets First Thing today | 05:17 |
| mwhudson | lifeless: um, it's extracted from the apache logs on the frontend | 05:17 |
| lifeless | theres no squid yet? | 05:17 |
| lifeless | if there was, we'd need a separate graph. | 05:18 |
| mwhudson | no | 05:18 |
| mwhudson | well, at least if there is, noone's told me about it | 05:18 |
| poolie | lifeless, i believe the critical incident policy tells you who's meant to follow up etc | 05:20 |
| lifeless | hmm, lpstats taking -forever- | 05:21 |
| lifeless | mwhudson: I can't see that graph | 05:22 |
| lifeless | mwhudson: can you just tell me ? | 05:22 |
| mwhudson | lifeless: "less than 1 req/s" would seem to be the summary | 05:24 |
| lifeless | would measuring in minutes be better ? | 05:24 |
| lifeless | 'd like a >1 figure, so I can do math. | 05:25 |
| mwhudson | lifeless: between 80 and 180 per 5 minutes | 05:25 |
| mwhudson | so that's, errrrrrrr, 16-36 per minute? | 05:25 |
| lifeless | 1/4-1/2 per second | 05:25 |
| mwhudson | the lack of robots.txt pushed it to ~1 per second and everything fell over | 05:26 |
| lifeless | I have a theory | 05:26 |
| lifeless | actually, robots.txt hit a bad url on many branches | 05:27 |
| lifeless | and that pushed it over | 05:27 |
| lifeless | http://trac.pythonpaste.org/pythonpaste/ticket/416 | 05:27 |
| mwhudson | yeah, annotate of files with deep history would be a good one to pound if you want to take codebrowse out | 05:30 |
| mwhudson | lifeless: huh! | 05:31 |
| mwhudson | lifeless: the thread killing does work at least sometimes | 05:31 |
| lifeless | so I'd rather say 'lh handled 4 times the load fine, except that we have a bug on some urls' | 05:31 |
| lifeless | mwhudson: possibly with low-valued threadids, or something. | 05:31 |
| mwhudson | this is another thing we should do, of course: change the default view from the filelisting to be a simple listing, not annotate | 05:32 |
| lifeless | mwhudson: is there a bug open on that ? | 05:33 |
| lifeless | if now, can you -please- open | 05:34 |
| lifeless | also, are you happy with all of johns work landing on LP ? | 05:34 |
| mwhudson | yes, but we should try to pre-seed the history.db for a few select large branches first | 05:34 |
| mwhudson | like launchpad, mysql, the kernel | 05:34 |
| lifeless | I'm mainly checking we don't need a seperate 'for lp' branch for Max | 05:35 |
| mwhudson | lifeless: t | 05:35 |
| mwhudson | grr | 05:35 |
| mwhudson | https://bugs.edge.launchpad.net/loggerhead/+bug/568148 | 05:35 |
| mup | Bug #568148: Default view for a file should be its content <performance> <loggerhead:Confirmed> <https://launchpad.net/bugs/568148> | 05:35 |
| lifeless | thumper: ping | 06:31 |
| === almaisan-away is now known as al-maisan | ||
| thumper | lifeless: pong (although leaving the office to eat) | 07:22 |
| lifeless | thumper: hi | 07:22 |
| lifeless | so | 07:22 |
| thumper | lifeless: leave a message and I'll get back to you as soon as possible :) | 07:22 |
| lifeless | I had a bug asking about getting allist of broken-due-to-upgraded-trunk branches | 07:23 |
| lifeless | you've closed the bug | 07:23 |
| lifeless | should I file a fresh dedicated one ? | 07:23 |
| thumper | yes | 07:23 |
| lifeless | ok will do | 07:24 |
| zyga | hmm | 08:21 |
| zyga | https://blueprints.edge.launchpad.net/ubuntu-arm/+spec/arm-m-validation-dashboard | 08:21 |
| zyga | (it was there a moment ago, now it's an OOPS) | 08:21 |
| zyga | OOPS-1607ED337 | 08:21 |
| zyga | can anyone help with this please? | 08:22 |
| zyga | bah, ok - the spec was renamed, sorry for causing noise | 08:22 |
| zyga | (still - the search did link to the wrong place) | 08:22 |
| mrevell | Morning | 08:57 |
| maxb | Ursinha-afk: Have you considered running your bugbot as a separate LP person, so that it is obvious that its actions are automated? | 10:51 |
| deryck | Morning, all. | 11:09 |
| wgrant | bigjools: AAAAAAA. https://edge.launchpad.net/ubuntu/+source/vowpal-wabbit/4.1+20100420-1/+build/1755930 is scary on several levels. | 11:13 |
| bigjools | yes, we've seen it | 11:14 |
| maxb | "special" :-) | 11:15 |
| bigjools | I thought we'd fixed this by stopping queue-builder :/ | 11:15 |
| wgrant | We must have fixed at least 5 of this sort of bug in the last six months. | 11:15 |
| wgrant | At least it will become impossible soon. | 11:15 |
| bigjools | well "fixed" is a strong word here :) | 11:15 |
| wgrant | For that particular one, yes. | 11:16 |
| bigjools | wgrant: so | 11:54 |
| bigjools | it's the result of a give-back | 11:55 |
| bigjools | why the builder does that instead of failing the build is somewhat odd | 11:55 |
| zyga | is there any SQL query that can extract a small subset of a big text field in a more efficient way (preferably without having to copy the whole field form the db to the client) | 11:55 |
| wgrant | bigjools: Yeah, I worked that out quite a while ago. | 11:56 |
| zyga | assuming I know the start/end offsets (in bytes/characters) | 11:56 |
| wgrant | It's the 'Illegal instruction' in the log. | 11:56 |
| wgrant | I forgot that we just fixed it so that it doesn't break b-m; the display is still all broken. | 11:56 |
| bigjools | I need to kill that build anyway | 11:56 |
| bigjools | I think that problem should also be a failed build, not a give-back | 11:56 |
| wgrant | Maybe it expects that some builders might be able to build some stuff. | 11:57 |
| wgrant | (like, say, armel) | 11:57 |
| wgrant | But that's an utterly stupid way of arranging that. | 11:57 |
| wgrant | So, yes, a quick deletion of that line will fix things. | 11:57 |
| wgrant | I can't think of any practical legit uses. | 11:57 |
| === mrevell is now known as mrevell-lunch | ||
| === matsubara-afk is now known as matsubara | ||
| === salgado-afk is now known as salgado | ||
| === mrevell-lunch is now known as mrevell | ||
| deryck | sinzui, please feel free to remove the bugs test, if the same condition is tested in test_securitycontact.py | 13:54 |
| sinzui | deryck, I did. I unconsciously knew it was a duplicate. It was a simple verification of the form label and that the form works. I wonder why it passed on ec2? | 13:56 |
| deryck | yeah, that seems odd that it would pass there. not sure why. | 13:56 |
| === Ursinha-afk is now known as Ursinha | ||
| === matsubara is now known as matsubara-lunch | ||
| === salgado is now known as salgado-lunch | ||
| === matsubara-lunch is now known as matsubara | ||
| === beuno is now known as beuno-lunch | ||
| Ursinha | bigjools, hi | 17:34 |
| bigjools | Ursinha: heyu | 17:35 |
| bigjools | or something that might be spelled better | 17:35 |
| Ursinha | bigjools, :) | 17:35 |
| Ursinha | bigjools, I have an oops and think you can help: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1600E1649 | 17:35 |
| bigjools | Ursinha: that's the same as the one you told me about a while ago | 17:36 |
| bigjools | Ursinha: noodles775 is looking into it, feel free to remind him :) | 17:36 |
| Ursinha | hehe | 17:37 |
| Ursinha | hahahahaha | 17:37 |
| Ursinha | bigjools, he heard you | 17:37 |
| bigjools | Ursinha: actually didn't you already file a bug about it? I think he commented on there | 17:38 |
| Ursinha | bigjools, I filed a bug that is a NotOneError | 17:39 |
| bigjools | it's basically down to a previous bug where sun-java6 ended up in partner and main at the same time | 17:39 |
| Ursinha | just found it | 17:39 |
| Ursinha | bug 580181 | 17:39 |
| mup | Bug #580181: DistributionSourcePackageRelease page still oopsing with NotOneError <oops> <Soyuz:Triaged> <https://launchpad.net/bugs/580181> | 17:39 |
| Ursinha | bigjools, OOPS-1600E1649 is a LocationError | 17:40 |
| Ursinha | are they the same? | 17:40 |
| bigjools | Ursinha: it's essentially the same underlying problem | 17:41 |
| bigjools | if you could reference this oops on that bug it would be helpful | 17:41 |
| Ursinha | bigjools, surely | 17:41 |
| bigjools | "Past week count: 686" | 17:41 |
| bigjools | eek | 17:41 |
| Ursinha | bigjools, if noodles785 could fix it, it would be helpful as well :) | 17:41 |
| bigjools | :) | 17:42 |
| Ursinha | bigjools, yesterday's count: 701 | 17:42 |
| bigjools | hmm no referrer | 17:42 |
| noodles785 | bigjools, Ursinha: I haven't been looking into it... I just did some initial investigation, but can look into it tomorrow. | 17:42 |
| Ursinha | noodles785, that would be appreciated, thanks :) | 17:42 |
| bigjools | noodles785: cheers | 17:43 |
| === noodles785 is now known as noodles775 | ||
| Ursinha | bigjools, thanks :) | 17:52 |
| bigjools | Ursinha: any time | 17:52 |
| === salgado-lunch is now known as salgado | ||
| mrevell | Righto peoples, I'm off. Ta ra. | 18:09 |
| === beuno-lunch is now known as beuno | ||
| === al-maisan is now known as almaisan-away | ||
| Alkini | how active is the search for a Launchpad Web Engineer (http://webapps.ubuntu.com/employment/canonical_LPWE/)? | 19:41 |
| beuno | Alkini, what do you mean? | 19:42 |
| Alkini | has the position just been sitting around for six months? or are a dozen people a day being considered? | 19:43 |
| beuno | Alkini, it's been opened up recently | 19:45 |
| beuno | and some people are being interviewed | 19:45 |
| beuno | not super sure what you're getting at :) | 19:45 |
| Alkini | I just applied today and don't know much beyond the "you might not hear back from us for three weeks" email so I just thought I'd ask :-) | 19:46 |
| beuno | well, it takes a day or two for the CV to be passed on from HR | 19:46 |
| beuno | and then it depends on the availability of the team lead | 19:47 |
| Alkini | sure, totally understandbale; I didn't mean to be impatient, just curious | 19:47 |
| beuno | I'm sure it's a position eager to be filled, so it'll probably be sooner than later | 19:47 |
| Alkini | alright, cool | 19:48 |
| Alkini | it's, presumably, the same story for the software engineer reporting to the launchpad code team lead? (http://webapps.ubuntu.com/employment/canonical_LP-SEC/) | 19:55 |
| beuno | well, different position, different people involved, but same HR process | 19:55 |
| Alkini | right | 19:56 |
| Alkini | you're on the ubuntu one team these days? | 20:00 |
| beuno | Alkini, yes I am, it's been a good 3 or 4 months now | 20:02 |
| beuno | still love Launchpad though :) | 20:02 |
| Alkini | heh | 20:02 |
| === matsubara is now known as matsubara-afk | ||
| === salgado is now known as salgado-afk | ||
| === Ursinha is now known as Ursinha-afk | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!