| wallyworld_ | thumper: should i ask sk to review those 2 daily recipe build list page branches? | 00:09 |
|---|---|---|
| poolie | bugclient now fails with | 01:04 |
| poolie | AttributeError: 'Entry' object has no attribute 'markAsDuplicate' | 01:04 |
| poolie | did this recently change in the api? | 01:04 |
| lifeless | what api version are you using? | 01:05 |
| lifeless | ?11958 | 01:06 |
| james_w | lifeless, api/1.0> why not? | 01:10 |
| lifeless | james_w: ? | 01:11 |
| poolie | bug 11958 | 01:11 |
| _mup_ | Bug #11958: Unable to show hidden files <nautilus (Ubuntu):Fix Released by seb128> <https://launchpad.net/bugs/11958> | 01:11 |
| lifeless | oh, that was a typo | 01:11 |
| lifeless | pasted to the wrong window | 01:11 |
| james_w | lifeless, <lifeless> salgado-afk: please don't put the blueprints in api/1.0 | 01:12 |
| lifeless | james_w: right. Because 1.0 is supported for many more years. | 01:12 |
| lifeless | but jml has the issue tracker thing in train for the next year. | 01:12 |
| james_w | why not stop the default being to add features to old versions then? | 01:13 |
| lifeless | there's a bug open on that | 01:13 |
| lifeless | IIRC | 01:13 |
| james_w | ok | 01:13 |
| lifeless | if there isn't, there should be - this has been discussed. | 01:13 |
| lifeless | we shouldn't 'support' things we haven't finished. | 01:14 |
| lifeless | we should do best effort etc | 01:14 |
| lifeless | but we need to allow room for mistakes. | 01:14 |
| poolie | hello james_w | 01:15 |
| james_w | hi poolie | 01:21 |
| lifeless | poolie: hi, so what api version are you using? | 01:26 |
| poolie | i don't know | 01:26 |
| lifeless | poolie: I ask, because its possible some things that were accidentally exposed have been shuffled, but we're meant to be very conservative about removing stuff.... trying to estimate whether to dig deeper or not. | 01:27 |
| poolie | i seem to be just taking the default? | 01:27 |
| poolie | <bug at https://api.edge.launchpad.net/1.0/bugs/415936> | 01:28 |
| _mup_ | Bug #415936: Merge into new branch produces strange log <Bazaar:New> <https://launchpad.net/bugs/415936> | 01:28 |
| poolie | so 1.0, i guess | 01:28 |
| lifeless | hmm, you're using edge too :) | 01:29 |
| lifeless | could you switch to LPNET_SERVICE_ROOT :) | 01:29 |
| poolie | heh, it's a hangover from that once being the only place to get it | 01:30 |
| poolie | much better to switch things on/off on the server | 01:30 |
| poolie | sure | 01:30 |
| lifeless | we can't sadly | 01:30 |
| lifeless | lp API clients start with a POST | 01:30 |
| poolie | i mean in future | 01:30 |
| lifeless | and blow up if thats redirected or otherwise handled by non-edge. | 01:30 |
| poolie | sure | 01:31 |
| lifeless | uhm, so 1.0 | 01:31 |
| lifeless | lets see | 01:31 |
| poolie | we could change lplib to make EDGE_SERVICE_ROOT == LPNET_SERVICE_ROOT | 01:31 |
| poolie | but that might just cause confusion | 01:31 |
| lifeless | we're going to do something like that | 01:31 |
| lifeless | perhaps with a deprecation warning | 01:31 |
| poolie | hm that name doesn't exist | 01:31 |
| poolie | ok, i have it | 01:32 |
| lifeless | ah cool | 01:33 |
| lifeless | what was it ? | 01:33 |
| lifeless | ok, so markAsDuplicate is in beta | 01:34 |
| lifeless | not in 1.0 or devel | 01:34 |
| lifeless | I can check the changelog but my guess is that you've moved from a launchpadlib that used beta by default to one that uses 1.0 by default | 01:35 |
| lifeless | the current idiom is to use duplicate_of = xxx | 01:35 |
| poolie | hm | 01:35 |
| poolie | something like that | 01:35 |
| lifeless | you can pass beta as the api version you want | 01:35 |
| poolie | though, this is on my maverick machine, running the packaged lplib | 01:35 |
| lifeless | or update your code | 01:35 |
| poolie | i doubt that changed in the last few weeks? | 01:36 |
| lifeless | that would be unusual. | 01:36 |
| lifeless | could you file a bug? Probably a mistake then. | 01:36 |
| LPCIBot | Yippie, build fixed! | 01:36 |
| LPCIBot | Project devel build (238): FIXED in 3 hr 45 min: https://hudson.wedontsleep.org/job/devel/238/ | 01:36 |
| LPCIBot | * Launchpad Patch Queue Manager: [r=abentley][ui=none][no-qa] New "images" command for bin/ec2 to | 01:36 |
| LPCIBot | display all current test images. | 01:36 |
| LPCIBot | * Launchpad Patch Queue Manager: [r=flacoste][ui=none][bug=677305] Downgrade bzr to 2.2.0 | 01:36 |
| poolie | hm, i see i was repeating myself and defaulting to edge in two places | 01:38 |
| lifeless | you might find https://dev.launchpad.net/PolicyAndProcess/OptionalReviews interesting | 01:38 |
| poolie | i've followed some of the mail about it | 01:39 |
| poolie | ok, so even on 1.0 non-edge, it still fails | 01:40 |
| lifeless | markAsDuplicate isn't in 1.0 | 01:40 |
| poolie | "stable interfaces in python are hard" | 01:40 |
| lifeless | perhaps it should be ;) - thus a bug is needed. | 01:40 |
| poolie | i'm pretty sure this was working a week or two ago | 01:40 |
| poolie | maybe a bit more than this, but within the last couple of months | 01:41 |
| poolie | i will file | 01:41 |
| poolie | the shorter Optional Reviews report: "data, bitches!" | 01:44 |
| poolie | i think that's great | 01:44 |
| poolie | to be devil's advocate | 01:44 |
| poolie | i think the previous experience is not so much showing everything needs review | 01:45 |
| poolie | but rather that people will use these to route around a broken review process | 01:45 |
| poolie | cf john's mail | 01:45 |
| poolie | but perhaps things have now changed | 01:45 |
| poolie | s//probably | 01:45 |
| lifeless | well | 01:50 |
| lifeless | I think that route around is better than stockpiling | 01:51 |
| thumper | wallyworld_: yes, fling them StevenK's way | 01:52 |
| thumper | wallyworld_: I'm just about to head and collect the girls from school | 01:52 |
| wallyworld_ | thumper: ok | 01:52 |
| thumper | wallyworld_: we should have a chat after that | 01:52 |
| wallyworld_ | thumper: i'll be here | 01:52 |
| poolie | lifeless: i think in the first version of this code i did assign to 'duplicate_of', but.. | 01:54 |
| poolie | that didn't work | 01:54 |
| poolie | i can't remember if the change was not saved, or if it gave an error | 01:54 |
| lifeless | wgrant: yo | 02:01 |
| lifeless | poolie: you need to call obj.lp_save() | 02:01 |
| poolie | i know that, and i think it wasn't enough | 02:02 |
| poolie | imbw | 02:02 |
| poolie | anyhow, bug 680339 | 02:02 |
| _mup_ | Bug #680339: 'Entry' object has no attribute 'markAsDuplicate' <Launchpad itself:New> <https://launchpad.net/bugs/680339> | 02:02 |
| lifeless | thank you | 02:03 |
| wallyworld_ | thumper: back in 15 mins - have to drop the kid to his McJob | 02:16 |
| thumper | wallyworld_: ok | 02:17 |
| poolie | thumper, wallyworld_, we're going to do a bzr 2.2.2 release at the end of this week | 02:20 |
| poolie | that should address the problems you hit in 2.2.1 | 02:20 |
| thumper | poolie: ok | 02:21 |
| poolie | feedback welcome, as always | 02:21 |
| wgrant | lifeless: Hi. | 02:22 |
| wgrant | What's broken? | 02:23 |
| lifeless | wgrant: cesium - removing cowboys | 02:24 |
| lifeless | wondering if you know the rev that landed the fix | 02:25 |
| lifeless | wgrant: how was your exam? | 02:25 |
| wgrant | lifeless: Exam was rather better than expected. | 02:25 |
| wgrant | cesium... what was the cowboy? The builder disabling thing? | 02:25 |
| spm | wgrant: all done? wooo! | 02:25 |
| wgrant | Indeed. | 02:26 |
| wgrant | lifeless: Unless there was more than just logging and not disabling builders, devel r11938 looks to be the one. | 02:30 |
| wallyworld_ | thumper: call now? | 02:35 |
| lifeless | wgrant: there was | 02:37 |
| lifeless | disabling the failure checking | 02:37 |
| lifeless | http://pastebin.com/Kr44BkbD | 02:38 |
| wgrant | lifeless: 11938 should render that pointless. | 02:39 |
| wgrant | Though I'd, er, check with someone else. | 02:39 |
| lifeless | wgrant: it looks complete to me | 02:41 |
| lifeless | the only gap is that first hunk | 02:43 |
| wgrant | Yeah, that's what I meant. | 02:44 |
| wgrant | I think. | 02:44 |
| lifeless | jamesh: do you remember how to tell zope to use a different thread count? | 03:57 |
| jamesh | lifeless: there is an option in the .conf file | 03:57 |
| jamesh | I think it might even be called thread_count | 03:57 |
| jamesh | this is the ZConfig .conf file | 03:59 |
| lifeless | is that 'launchpad.conf' ? | 03:59 |
| jamesh | yes | 04:05 |
| jamesh | lifeless: looking at one of the ancient launchpad trees on my system, one of the launchpad.conf files under configs/ has "threads 16" at the top level | 04:10 |
| lifeless | thanks! | 04:10 |
| lifeless | also, terrible idea :) | 04:10 |
| jamesh | I remembered that we had bumped the count up for the demo.launchpad.net instance | 04:11 |
| jamesh | so checked that config | 04:11 |
| jamesh | that instance wasn't under heavy load, but it was easy for one user to block everyone else with the default thread count | 04:12 |
| lifeless | yeah, | 04:18 |
| lifeless | we're in the process of (with measurement) dropping down to 1 thread per appserver | 04:18 |
| poolie | lifeless: interesting! i don't recall you posting about it | 04:32 |
| poolie | (not that you have to, i suppose) | 04:32 |
| lifeless | in various perf tuesday mails | 04:32 |
| poolie | on the grounds that, if they're not wasting time, you wouldn't get more than one python thread to run anyhow? | 04:32 |
| lifeless | we're seeing things that thread starvation is the best explanation for | 04:32 |
| lifeless | and yes | 04:32 |
| lifeless | that too | 04:32 |
| poolie | ok, i do see a mention in passing | 04:33 |
| lifeless | jamesh: yes, thats it | 04:35 |
| lifeless | jamesh: thanks for finding it | 04:35 |
| lifeless | poolie: we could reasonable expect total time/database time threads to be a reasonable figure | 04:36 |
| lifeless | but its simpler to let the OS manage things at that point | 04:36 |
| lifeless | + if we do decide to debug something only one user will be impacted | 04:37 |
| lifeless | poolie: I did some more analysis in a ubuntu-one thread | 04:37 |
| poolie | agree about letting the OS do it | 04:39 |
| poolie | istm the main drawback is there are things that python knows are shareable in memory, that the OS doesn't | 04:40 |
| poolie | like modules | 04:40 |
| poolie | and for lp they tend to be large | 04:40 |
| poolie | but maybe this is not a sufficiently important factor | 04:40 |
| lifeless | its a couple hundred MB | 04:42 |
| lifeless | 1.6GB on a fully loaded machine. | 04:42 |
| lifeless | it could be better | 04:42 |
| lifeless | but we should get a win regardless, or so says the theory. | 04:43 |
| lifeless | yay | 04:46 |
| lifeless | https://launchpad.net/~lifeless/+commentedbugs working | 04:46 |
| poolie | timed out for me.. | 04:47 |
| lifeless | hahrugle | 04:47 |
| poolie | in 'select count from bugtask....' | 04:48 |
| poolie | but it's a lovely sounding url :) | 04:48 |
| lifeless | what revno | 04:48 |
| poolie | 11952 | 04:48 |
| lifeless | damn | 04:48 |
| lifeless | At least 42 queries/external actions issued in 2.16 seconds | 04:48 |
| lifeless | r11952 | 04:48 |
| lifeless | poolie: its working consistently for me | 04:50 |
| lifeless | poolie: whats the OOPS id ? | 04:50 |
| poolie | https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1788K350 | 04:52 |
| poolie | works for me now | 04:52 |
| poolie | is this a new feature, or just a newly-faster feature? | 04:53 |
| poolie | worked this time | 04:53 |
| lifeless | top OOPS since the monthly rollout | 04:53 |
| poolie | ah i see | 04:53 |
| poolie | maybe it was 'want an affecting bugs list' i was thinking of | 04:53 |
| spm | wgrant: btw. I'm bemused and entertained at your dedication. Finish final exam, hop on IRC. ISTR that post my final exam at Uni, a group of us went to the Brekky Creek Hotel in Brisvegas for a Steak lunch and afternoon of entertaining the regulars in the beer garden with (badly) Monty Python songs sung. | 05:59 |
| wgrant | spm: Well, a few of us finished early, went to pub, consumed beer, returned to the exam venue to see everyone else, then went back to my office. | 06:01 |
| wgrant | And then IRC, yes :P | 06:01 |
| spm | bwhahaha | 06:02 |
| lifeless | spm: hey | 06:16 |
| lifeless | sorry, ECHAN | 06:16 |
| poolie | spm, ah, the brekky creek | 06:42 |
| poolie | is it a known problem that all api calls against staging are giving 500 errors? | 06:42 |
| poolie | and indeed the web ui too | 06:43 |
| stub | Hayfever or cold :-( Suspect today is another sick day. | 06:44 |
| spm | yeah, staging is borked atm. been fixing the borked prod rollout first tho | 06:44 |
| poolie | heh, that's probably a good choice | 06:45 |
| poolie | :) thanks spm | 06:45 |
| === almaisan-away is now known as al-maisan | ||
| adeuring | good morning | 08:41 |
| bigjools | morning | 08:41 |
| bigjools | how was the exam wgrant? | 08:42 |
| lifeless | is there someone around that can qa https://bugs.launchpad.net/rosetta/+bug/669831 ? | 08:54 |
| _mup_ | Bug #669831: obsolete translations exported to the branch <code-integration> <qa-needstesting> <Launchpad Translations:Fix Committed by danilo> <https://launchpad.net/bugs/669831> | 08:54 |
| mrevell | Morning | 09:09 |
| bigjools | lifeless: I've landed all code to cover the cowboys on cesium, please roll it out (or I can) | 09:12 |
| lifeless | bigjools: its done | 09:12 |
| bigjools | lifeless: ah great, you checked or Picarded it? | 09:12 |
| lifeless | spm deployed | 09:12 |
| lifeless | and noone has screamed yet | 09:12 |
| lifeless | I figured I'd chat with you briefly :) | 09:13 |
| bigjools | heh | 09:13 |
| * bigjools looks at log | 09:13 | |
| bigjools | lifeless: I thought it was in the nodowntime set? | 09:14 |
| lifeless | bigjools: it was taken out when it blew up | 09:15 |
| lifeless | when things get cowboyed we remove them from nodowntime. | 09:15 |
| bigjools | so it's back in? I thought I said I was going to approve that first ... | 09:16 |
| lifeless | we may be crossing wires | 09:16 |
| lifeless | back at uds it wasn't in, and we discussed what it would take to be in | 09:16 |
| lifeless | that you were cc'd on a discussion and gave your blessing. | 09:16 |
| lifeless | then the big branch blew builders away | 09:17 |
| lifeless | so it was cowboy-fixed and removed [to stop deploys uncowboying it] | 09:17 |
| lifeless | we cross checked that you'd landed fixes [and they were to be deployed] today, and so uncowboyed it (which implies putting it back in nodowntime] | 09:17 |
| lifeless | bigjools: if you had wanted a further cross-check with you, I'm really sorry - I didn't realise. | 09:18 |
| bigjools | lifeless: np, I talked with Tom not you | 09:19 |
| bigjools | as it happens it's fine to roll, but that's because I had landed everything | 09:19 |
| lifeless | right, we looked first ;) | 09:19 |
| lifeless | in future, if you want a cross-check, could you note that on LPS against the cowboy, or the DeploymentException ? | 09:19 |
| bigjools | yes but one of the cowboys is not landed, but deliberately. How did you reconcile that? :) | 09:20 |
| lifeless | bigjools: knowledge. | 09:20 |
| bigjools | :) | 09:20 |
| lifeless | I had discussed the fault with you. | 09:20 |
| lifeless | so I knew you put it in while you diagnosed the root cause. | 09:20 |
| bigjools | we need to figure out wtf the builders are taking >30 seconds to accept connections | 09:20 |
| lifeless | I also included that cowboy as a ready to go patch in a mail to losas | 09:21 |
| lifeless | so if there is a submarine present there | 09:21 |
| lifeless | its at-hand to recowboy | 09:21 |
| bigjools | one of the side effects of that cowboy is that jobs will "stick" on genuinely failed builders | 09:22 |
| bigjools | I have to keep checking the log | 09:22 |
| bigjools | lifeless: so, did we talk about figuring out the massive slave connection delays? | 09:30 |
| lifeless | not as such | 09:31 |
| henninge | hey jtv! ;) | 09:36 |
| jtv | hi henninge! | 09:36 |
| henninge | jtv: let's be chatty ;) | 09:36 |
| jtv | henninge: I don't think this connection will support voice chat. | 09:37 |
| henninge | jtv: oic | 09:38 |
| henninge | you are also not on #translations | 09:38 |
| jtv | just a mo' | 09:38 |
| lifeless | jtv: hi | 09:40 |
| jtv | hi | 09:40 |
| lifeless | https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html | 09:40 |
| jtv | uh-oh | 09:41 |
| lifeless | are you able to qa 669831 on [qa]staging ? | 09:41 |
| jtv | henninge: hang on, lifeless wants something :) | 09:41 |
| lifeless | no panic | 09:41 |
| jtv | lifeless: very slow connection, so will be a bit slow to respond | 09:41 |
| lifeless | but 11960 would remove another cowboy in the datacentre | 09:41 |
| lifeless | jtv: like I say | 09:42 |
| lifeless | no panic | 09:42 |
| henninge | jtv, lifeless: I can do that | 09:42 |
| lifeless | awesome | 09:42 |
| lifeless | ok, gnight all | 09:42 |
| jtv | g'night | 09:42 |
| henninge | lifeless: good night | 09:42 |
| * jtv runs around a bit and screams for a while, just because he was told not to panic | 09:42 | |
| jtv | it's the principle of the thing | 09:42 |
| bigjools | have you considered working on Soyuz? | 09:53 |
| wgrant | bigjools: Better than expected. | 10:06 |
| bigjools | wgrant: excellent, but shouldn't you be out celebrating now | 10:07 |
| wgrant | Did that earlier :P | 10:07 |
| wgrant | But yes, probably. | 10:07 |
| bigjools | you party animal | 10:08 |
| bigjools | did you get a chance to look at the expiration query? | 10:08 |
| wgrant | Heh. | 10:08 |
| wgrant | Will look now. | 10:08 |
| wgrant | bigjools: Where's the latest version of that query? | 10:11 |
| bigjools | the one I pasted :) | 10:11 |
| wgrant | k | 10:11 |
| bigjools | http://pastebin.ubuntu.com/535167/ | 10:12 |
| wgrant | Yup. | 10:12 |
| wgrant | bigjools: Are the LFC, DS and DAS joins in the EXCEPT completely useless, or am I stupid and blind? | 10:21 |
| bigjools | one sec | 10:22 |
| wgrant | bigjools: http://pastebin.ubuntu.com/535491/ | 10:24 |
| wgrant | Should be equivalent, except with the retention condition fixed. | 10:25 |
| wgrant | (now it will exclude if the file is unremoved or Obsolete, rather than Published or Obsolete) | 10:25 |
| wgrant | Which should make p-d-r less sad. | 10:25 |
| wgrant | Hmm. | 10:27 |
| wgrant | bigjools: Did the domination thingy help dogfood at all? I guess it pales in comparison with the file list generation :/ | 10:29 |
| bigjools | wgrant: yes, file lists take ~4 hours | 10:29 |
| bigjools | for just maverick release pocket :/ | 10:29 |
| bigjools | something has regressed a lot | 10:29 |
| bigjools | it used to take ~30 mins | 10:29 |
| wgrant | I will poke it in the eye in a few weeks. | 10:29 |
| bigjools | DF is slow, but ... | 10:30 |
| wgrant | Heh. | 10:30 |
| bigjools | wgrant: sql looks good, I am trying it on DF | 10:43 |
| wgrant | bigjools: So, about those people who feel like they need to use obsolete PPAs... | 10:47 |
| bigjools | OEM | 10:47 |
| wgrant | How did I guess :( | 10:48 |
| bigjools | now, I am struggling to understand wtf builder sometimes take in excess of 180 seconds to accept a connection from the manager | 10:49 |
| * bigjools -> caffeine | 10:50 | |
| wgrant | bigjools: Load graphs from non-virt builders pls. | 10:51 |
| wgrant | It might at least give us some idea of if that's the issue. | 10:51 |
| === matsubara-afk is now known as matsubara | ||
| bigjools | jml: remember my dirty reactor failure in my b-m tests? | 11:14 |
| bigjools | I added debug output on the Deferreds and it's caused by distribution_mirrorprober. There's a test isolation error... :/ | 11:16 |
| wgrant | The reactor is shared‽ | 11:17 |
| bigjools | in tests | 11:17 |
| wgrant | My interrobang stands. | 11:17 |
| * jml briefly steps in from the sick room | 11:17 | |
| bigjools | man flu? | 11:18 |
| jml | wgrant: yeah, there's exactly one reactor. | 11:18 |
| jml | wgrant: it's arguably the biggest flaw in Twisted | 11:18 |
| wgrant | jml: This sounds... like Zope. | 11:18 |
| bigjools | it would be kinda hard to have more than one | 11:19 |
| jml | bigjools: I know about those tests. my testtools-experiment branch fixes those delayed calls. | 11:19 |
| jml | bigjools: but it's blocked on landing by some weird-ass 500 from the librarian | 11:19 |
| bigjools | jml: yay. So, what can I do about it in my branch? | 11:19 |
| jml | bigjools: those tests need to be completely rewritten... let me find you my workaround | 11:20 |
| bigjools | and why are they leaking to my test? | 11:20 |
| jml | bigjools: because the distributionmirror_prober tests aren't using trial | 11:20 |
| bigjools | ah ... | 11:20 |
| jml | bigjools: so the calls are going on to the reactor, and when your tests are cleaned up ... bang | 11:20 |
| bigjools | that's kinda bad from a test isolation PoV :/ | 11:21 |
| bigjools | wgrant: | 11:21 |
| bigjools | total_files | space_saved | 11:21 |
| bigjools | -------------+-------------- | 11:21 |
| bigjools | 184899 | 307432516434 | 11:21 |
| jml | bigjools: yes. the problem is the mutable global state that is the reactor | 11:22 |
| bigjools | indeed | 11:22 |
| jml | bigjools: got any suggestions on how to make it better? | 11:22 |
| wgrant | bigjools: Mm, not too implausible. | 11:22 |
| bigjools | jml: I'd make Trial clean the reactor when starting a test? | 11:22 |
| jml | bigjools: it can't do that. there might be in-process Twisted-using fixtures | 11:22 |
| jml | bigjools: http://pastebin.ubuntu.com/535507/ <- should work around the problem | 11:23 |
| bigjools | jml: at the very start of the test there should be no fixtures yet though? | 11:23 |
| jml | bigjools: not if they are shared between tests | 11:23 |
| bigjools | aieeee | 11:23 |
| bigjools | jml: I'll poke your workaround in, thanks | 11:24 |
| jml | wgrant: incidentally, it doesn't sound like Zope to me. | 11:24 |
| wgrant | jml: Opaque global state. | 11:24 |
| wgrant | The Zope Way. | 11:24 |
| bigjools | jml: if it's complaining about them when the test ends, how can a fixture's Deferreds never get in the way then? | 11:24 |
| bigjools | if it's shared between tests that is | 11:25 |
| jml | bigjools: good point. in Trial, there's some historical crap back from when we thought setUpClass would be a good idea | 11:25 |
| jml | (it's a terrible idea, we got rid of it, Guido added it back to Python again, and so the circle of crap continues) | 11:26 |
| bigjools | :/ | 11:26 |
| bigjools | so if someone uses setUpClass with a Deferred, your tests will always fail | 11:26 |
| jml | bigjools: in testtools, I guess we could clean the reactor before tests | 11:26 |
| jml | bigjools: no, Trial postpones checking of those things until tearDownClass runs. | 11:27 |
| jml | bigjools: *that's* the historical crap. | 11:27 |
| bigjools | oy | 11:27 |
| jml | as I said, clearing it out in testtools would help | 11:28 |
| bigjools | yeah | 11:29 |
| jml | although it wouldn't help that much. | 11:29 |
| jml | "some test before this one was bonkers" | 11:29 |
| jml | it's still an improvement over the current situation | 11:29 |
| bigjools | I won't ask why the mirrorprober tests are not using Trial then.... :) | 11:30 |
| jml | I have NFI | 11:30 |
| jml | if I have my way, they'll be using testtools before the year is out. | 11:30 |
| bigjools | \o/ | 11:30 |
| bigjools | hmm I need to book some holiday | 11:31 |
| jml | anyway, all this excitement is threatening my delicate constitution | 11:31 |
| bigjools | jml: Berocca | 11:31 |
| jml | bigjools: it's a cold, not a hangover. :P | 11:31 |
| bigjools | jml: :) it still works | 11:31 |
| bigjools | the fizzy stuff | 11:31 |
| bigjools | get better soon anyway, go get some rest | 11:32 |
| * jml watches The Wire instead. | 11:32 | |
| bigjools | a friend of mine swears his PS3 is medicinal | 11:32 |
| jml | heh | 11:32 |
| === al-maisan is now known as almaisan-away | ||
| deryck | Morning, all. | 12:04 |
| bigjools | wgrant: so, I figured out the problem with the log parser | 12:23 |
| wgrant | bigjools: !! | 12:25 |
| wgrant | What is it? | 12:25 |
| bigjools | wgrant: it reads in gzip files in their entirety :/ | 12:26 |
| bigjools | see lp/services/apachelogparser/base.py | 12:26 |
| bigjools | get_fd_and_file_size() | 12:26 |
| bigjools | le heavy sigh | 12:26 |
| wgrant | bigjools: Hahaha. | 12:27 |
| wgrant | I thought gzip stored the uncompressed size in the header... | 12:28 |
| wgrant | Yes, it's in the footer. | 12:28 |
| wgrant | Not sure how we can access that from Python, though... | 12:29 |
| bigjools | but does any of the python module read that | 12:29 |
| wgrant | It's limited to 4 bytes, so it probably doesn't. | 12:33 |
| wgrant | But let's see. | 12:33 |
| bigjools | http://stackoverflow.com/questions/1704458/get-uncompressed-size-of-a-gz-file-in-python | 12:33 |
| wgrant | It looks like we probably have no choice but to read in chunks. | 12:34 |
| bigjools | did you notice the last answer on that page :) | 12:36 |
| wgrant | Suggesting len(fd.read())? | 12:36 |
| wgrant | Oh. | 12:37 |
| wgrant | Hahahah. | 12:37 |
| wgrant | I rarely look at the author... | 12:37 |
| bigjools | I am tempted to grab the last 4 bytes | 12:38 |
| wgrant | But 2**32... | 12:39 |
| wgrant | I guess we can make it explode if it ever tries to write a bytes_read greater than that. | 12:39 |
| wgrant | Since something is pretty broken if we have a 4GiB log file, I guess. | 12:39 |
| bigjools | they are in the region of 1.2G uncompressed | 12:40 |
| wgrant | Oh. | 12:41 |
| wgrant | That is inconvenient... | 12:41 |
| wgrant | Also, that's huge. | 12:41 |
| bigjools | it's fine | 12:41 |
| wgrant | It's far too close for my liking, but OK. | 12:42 |
| bigjools | we can put a limit on it | 12:43 |
| wgrant | Yes, but a limit within a couple of orders of magnitude of the current value seems like a really bad idea. | 12:44 |
| * bigjools tries dirty hack on DF | 12:47 | |
| bigjools | score | 13:04 |
| wgrant | It works? | 13:04 |
| bigjools | yes | 13:06 |
| wgrant | So I didn't break it after all :D | 13:07 |
| bigjools | seems so :) | 13:07 |
| bigjools | 2010-11-23 13:03:04 INFO Parsed 5000000 lines resulting in 16085 download stats. | 13:07 |
| wgrant | Hmm. | 13:08 |
| bigjools | I lied, one file is 2.9G | 13:08 |
| wgrant | Ow. | 13:09 |
| wgrant | bigjools: Could you sum all the BPRDC.count? | 13:10 |
| wgrant | Just to see that it actually handles most of the lines. | 13:10 |
| wgrant | Although I guess lots of those lines will be Packages/Sources, so it might not be too similar. | 13:11 |
| bigjools | 6880 | 13:12 |
| bigjools | it's still going though | 13:12 |
| bigjools | it will take a while | 13:12 |
| bigjools | and it's hammering DF | 13:12 |
| bigjools | good time for lunch, see you later | 13:13 |
| * wgrant sleeps. | 13:13 | |
| wgrant | Thanks for fixing that. | 13:13 |
| bigjools | my pleasure | 13:15 |
| jelmer | 'morning benji, abentley | 14:00 |
| abentley | jelmer: morning. | 14:00 |
| === almaisan-away is now known as al-maisan | ||
| benji | morning jelmer, or afternoon as it were ;) | 14:05 |
| === matsubara is now known as matsubara-lunch | ||
| jitu | how to add https://launchpad.net/~falk-t-j/+archive/lucid/+build/2018840 in repository? | 14:10 |
| jitu | anybody to help? | 14:10 |
| bigjools | jitu: follow the instructions here https://launchpad.net/~falk-t-j/+archive/lucid | 14:12 |
| bigjools | where it says "Adding this PPA to your system" | 14:12 |
| jitu | checking... | 14:14 |
| jitu | bigjools, thnx | 14:15 |
| deryck | mars, hi. You around? | 14:19 |
| deryck | Or maybe gary_poster could help me. gary_poster, ping? | 14:24 |
| gary_poster | hey deryck. what's up | 14:24 |
| deryck | Hi gary_poster. Does this revno look like the right way that I would disable windmill tests to you? http://bazaar.launchpad.net/~deryck/launchpad/rockstar-js-refresh/revision/11726 | 14:24 |
| * gary_poster is skeptical he will know, but is looking | 14:25 | |
| gary_poster | deryck: yeah, that looks like a very reasonable approach to me, especially if you have evidence it works ;-) . | 14:25 |
| deryck | gary_poster, heh. that's the problem, I don't. ;) Started an ec2 test run last night that disappeared. I assume something hung and the test was killed.... | 14:26 |
| gary_poster | :-P | 14:26 |
| deryck | so I started another run, but was looking for some confirmation that the patch looked right. :-) | 14:27 |
| gary_poster | deryck: I can assert that I believe that should have worked | 14:27 |
| deryck | gary_poster, good enough. Thanks! | 14:27 |
| gary_poster | :-) np | 14:27 |
| deryck | :-) | 14:27 |
| deryck | FWIW, my current test run seems to be going well. The tests started up much faster than earlier attempts I had. | 14:27 |
| mars | deryck, you may need '!(MailmanLayer|WindmillLayer)' instead, but I don't know for sure. | 14:28 |
| mars | empirical evidence needed | 14:29 |
| deryck | mars, gary_poster, also, I opened Bug #680497 about missing test coverage. If this didn't belong on Foundations, sorry. I wasn't sure. | 14:29 |
| _mup_ | Bug #680497: jstests for LP JavaScript client are not running automatically <Launchpad Foundations:New> <https://launchpad.net/bugs/680497> | 14:29 |
| deryck | mars, ok, I'll watch the run closely and see. Thanks! | 14:29 |
| deryck | if I see a windmill test go, I'll kill it ASAP and try again :-) | 14:30 |
| gary_poster | deryck: Foundations: yeah, close enough :-) . I might add the web group. | 14:30 |
| deryck | ok, cool | 14:30 |
| gary_poster | deryck, mars: mars makes a good point. I'm not sure if the layer thing is an and or an or. ./bin/test --help should say. looking | 14:31 |
| deryck | heh "is an and or an or" is hard to parse on two cups of coffee only | 14:31 |
| deryck | the hung run suggests my patch might not have worked. | 14:32 |
| gary_poster | deryck, :-) mars is right. "--layer" is a logical or. that is, it will run include all layers that are not Windmill ORed with all layers that are not Mailman, resulting in all layers. | 14:32 |
| gary_poster | s/run include/include | 14:33 |
| deryck | ah | 14:33 |
| deryck | gary_poster, so I need the form mars suggested then, right? | 14:33 |
| gary_poster | yes | 14:33 |
| deryck | ok, cool. an ec2 run killin' I shall go.... | 14:33 |
| gary_poster | :-) k | 14:33 |
| deryck | thanks mars and gary_poster! | 14:33 |
| gary_poster | np | 14:33 |
| * deryck dreads seeing the ec2 bill this month | 14:34 | |
| bigjools | jml: are you too ill to help a bit with the buildd-manager timeouts? My fix hasn't worked. | 14:49 |
| abentley | bigjools: Has lamont installed the new buildd (rev 74)? | 14:51 |
| bigjools | abentley: you should ask him, not me ;) | 14:52 |
| abentley | bigjools: because if he has, that's a bad sign, but if he hasn't, it could help with the timeouts. | 14:53 |
| === salgado is now known as salgado-lunch | ||
| === matsubara-lunch is now known as matsubara | ||
| jml | bigjools: I can try. Wassup? | 15:14 |
| bigjools | jml: that timeout stuff I added has had no effect :( | 15:14 |
| bigjools | here's an example of the sequence: | 15:15 |
| jml | bigjools: ok. so maybe we misdiagnosed the problem? | 15:15 |
| bigjools | 2010-11-23 14:46:27+0000 [QueryProtocol,client] Resuming hassium (http://hassium.. | 15:15 |
| bigjools | 2010-11-23 14:46:32+0000 [-] Asking builder on http://hassium.ppa:8221/filecache to ensure it has file chroot-ubuntu-lucid-i386.tar.bz2 | 15:15 |
| bigjools | 2010-11-23 14:46:54+0000 [Uninitialized] Scanning hassium failed with: TCP connection timed out: 110: Connection timed out | 15:15 |
| bigjools | the timeout is 22 seconds later | 15:15 |
| bigjools | not even the default 30 | 15:15 |
| bigjools | so I reckon misdiagnosis is quite probable | 15:16 |
| jml | bigjools: is your timeout stuff actually on the relevant machines? | 15:16 |
| bigjools | jml: yes, I had Tom do a paranoia grep | 15:18 |
| bigjools | I'm at a bit of a loss here | 15:18 |
| jml | poking at the code now | 15:19 |
| bigjools | we probably need some more debugging logging | 15:19 |
| jml | yes. and the ability to switch it on and off in runtime | 15:19 |
| jml | bigjools: do you have a traceback with that error? | 15:20 |
| bigjools | jml: no | 15:21 |
| jml | bigjools: isn't that unexpected? | 15:21 |
| bigjools | it's one of the "known" errors so it doesn't run the traceback. | 15:21 |
| jml | BuildSlaveFailure, CannotBuild, BuildBehaviorMismatch, | 15:21 |
| jml | CannotResumeHost, BuildDaemonError, CannotFetchFile): | 15:21 |
| jml | which one? | 15:21 |
| bigjools | it'll be getting re-raised somewhere | 15:23 |
| jml | bigjools: we should also remove some of the unnecessary layers of stack toot-sweet. We have to trawl through this code frequently enough that it's a noticeable cost. | 15:24 |
| bigjools | wtf is a toot-sweet? | 15:24 |
| jml | sorry, an expression from childhood. "very quickly" | 15:25 |
| bigjools | "tout de suite" | 15:25 |
| bigjools | :) | 15:25 |
| bigjools | but yeah, agree | 15:26 |
| jml | bigjools: I learnt it from folk with Middlesex accents. | 15:26 |
| jml | anyhow | 15:26 |
| bigjools | hah | 15:27 |
| jml | bigjools: my reading of the code shows that it's failing in startBuild (builder.py) after the call to resumeSlaveHost succeeds | 15:27 |
| jml | because resumeSlaveHost prepends crap to the error message, and we don't see that in the logs | 15:28 |
| jml | likewise, the error can't be coming from any of the eb_foo in startBuild, because they also mutate the error message | 15:28 |
| bigjools | correct | 15:28 |
| jml | ergo, it comes from resume_done | 15:28 |
| jml | or something after startBuild | 15:28 |
| bigjools | it will be the first call that tries to dispatch the chroot | 15:29 |
| bigjools | bearing in mind they are >600M I wonder if that is poking a subtle bug | 15:29 |
| jml | there doesn't seem to be anything after startBuild that does anything particularly interesting | 15:30 |
| jml | bigjools: do we know what type of job this is? | 15:30 |
| bigjools | binarypackage | 15:30 |
| jml | hmm | 15:32 |
| bigjools | something is catching that error and re-raising it | 15:34 |
| jml | that's, well, odd. | 15:34 |
| bigjools | as a known exception - otherwise we'd see a traceback | 15:34 |
| bigjools | actually - I wonder if it contains any trace info | 15:34 |
| jml | because, afaict, it's being raised by the second line in dispatchBuildToSlave in binarypackagebuildbehavior | 15:35 |
| jml | d = self._builder.slave.cacheFile(logger, chroot) | 15:35 |
| jml | which has absolutely no errback added to it until _scanFailed in manager.py | 15:35 |
| jml | (again, my reading only, not actually tested) | 15:36 |
| jml | actually, I'm going to say some stuff with triple-x markers so the conversation can be more readily grepped for actions | 15:36 |
| jml | XXX: change _scanFailed to have a different prefix to the log message for unexpected errors | 15:37 |
| jml | XXX: collapse some of the unnecessary indirection in builder.py (e.g. updateStatus, updateBuilderStatus; updateBuild; _dispatchBuildCandidate) | 15:38 |
| jml | bigjools: the other big question is why 22s | 15:38 |
| jml | bigjools: what are the values for timeouts in production? | 15:39 |
| bigjools | 180s | 15:39 |
| bigjools | I simply cannot fathom where 22 comes from | 15:39 |
| jml | perhaps we're using an unexpected config setting? | 15:39 |
| jml | perhaps it's 20 + noise? | 15:39 |
| bigjools | the delay on the failures in the log is not consistent either | 15:39 |
| jml | bigjools: what's the smallest? | 15:39 |
| bigjools | that's the smallest I've seen, but it's hard to grep | 15:39 |
| jml | since we've got blocking code in prod, it's to be expected that there'll be variation | 15:40 |
| bigjools | yep | 15:40 |
| bigjools | jml: those eb_ functions are not doing much any more | 15:41 |
| jml | bigjools: I can believe it | 15:42 |
| jml | hmm. | 15:42 |
| bigjools | they are used when doing the resume op :) | 15:43 |
| jml | bigjools: what's in the logs roughly 3 minutes before that error message? | 15:43 |
| bigjools | not a lot | 15:44 |
| jml | bigjools: do we get the same three lines every time the error happens? | 15:44 |
| bigjools | sorta | 15:44 |
| bigjools | they're obviously spread around the file | 15:44 |
| bigjools | but it's always the chroot dispatch as far as I've seen | 15:44 |
| jml | hmm. | 15:45 |
| jml | grepping twisted code reveals it's definitely a TCPTimedOutError | 15:45 |
| bigjools | I wonder if the slave is disconnecting before it's replied? | 15:45 |
| * jml looks up what ETIMEDOUT means | 15:46 | |
| bigjools | jml: heh, you know what, the timeout on the slave will still be 30 seconds, right? | 15:49 |
| jml | bigjools: why so? | 15:49 |
| bigjools | it's also twisted | 15:49 |
| jml | hmm. | 15:49 |
| jml | I don't think listening works in the same way | 15:50 |
| bigjools | it depends on what stupidity it has | 15:50 |
| jml | how is the buildd launched in production? | 15:50 |
| bigjools | it's part of init.d | 15:51 |
| jml | Timeout while attempting connection. The server may be too busy to accept new connections. Note that for IP sockets the timeout may be very long when syncookies are enabled on the server. | 15:53 |
| jml | hah | 16:00 |
| bigjools | is that a victorious hah? | 16:00 |
| jml | maybe. | 16:01 |
| bigjools | show me the beans | 16:01 |
| jml | so, if I understand correctly, the timeout passed to connectTCP does not in fact control the UNIX-level socket timeout | 16:01 |
| bigjools | it's a callLater to cancel the Deferred I think | 16:01 |
| jml | exactly | 16:01 |
| jml | but the TimeoutError that generates is different to TCPTimedOutError | 16:02 |
| bigjools | yep | 16:02 |
| jml | which is a mapping for ETIMEDOUT | 16:02 |
| bigjools | interesting | 16:02 |
| jml | IIUC, the call to socket.connect_ex() in t/internet/tcp.py is timing out | 16:03 |
| === beuno is now known as beuno-lunch | ||
| jml | unfortunately, I don't know whether the timeout is being set client-side or server-side, somewhere in Twisted, somewhere in Python or somewhere deeper | 16:04 |
| bigjools | maybe the stack trace will tell us when I get that cowboy in | 16:05 |
| bigjools | but I suspect not :/ | 16:05 |
| jml | bigjools: well, the timeout is being set in a different call | 16:05 |
| * jml flicks through APUE | 16:05 | |
| jml | nope | 16:12 |
| === deryck is now known as deryck[lunch] | ||
| === al-maisan is now known as almaisan-away | ||
| * bigjools otp for a bit | 16:30 | |
| === almaisan-away is now known as al-maisan | ||
| === rockstar` is now known as rockstar | ||
| lifeless | morning | 16:58 |
| lifeless | bigjools: how did cesium go ? | 16:58 |
| lifeless | henninge: hi; how did you go on 669831? | 16:59 |
| lifeless | bigjools: ah, found your mail. | 17:03 |
| bigjools | lifeless: yeah, badly | 17:07 |
| lifeless | I've suggested a feature flag to you, to let us get rid of the cowboy but keep the code path easily off/on as needed. | 17:08 |
| lifeless | (in mail) | 17:08 |
| bigjools | lifeless: +1 to the flag. But only if I don't work out the problem this week. | 17:11 |
| lifeless | sure | 17:12 |
| bigjools | maybe I'll do it along with an attempted fix actually | 17:12 |
| bigjools | lifeless: the thing that we discovered earlier is that the timeouts I changed are not firing, something else is. | 17:12 |
| bigjools | we see a TCPTimedOutError | 17:13 |
| bigjools | if it were the timeout value firing it would be a Deferred cancelled error | 17:13 |
| bigjools | the question is, where the heck is generating that | 17:13 |
| lifeless | do you generate an OOPS ? | 17:14 |
| lifeless | if so, its backtrace might help | 17:15 |
| bigjools | lifeless: there's no trace on the exception :( | 17:15 |
| bigjools | it should get logged, but there's nada | 17:15 |
| lifeless | grah | 17:16 |
| bigjools | well - there IS a traceback, it's just one line | 17:16 |
| lifeless | is it an OOPS ? | 17:16 |
| lifeless | or something else | 17:16 |
| bigjools | no, I don't generate an oops | 17:16 |
| lifeless | whats logging the error for you ? | 17:16 |
| bigjools | because they are routine failures | 17:16 |
| bigjools | my code! | 17:16 |
| bigjools | see _scanFailed() in lib/lp/buildmaster/manager.py | 17:17 |
| lifeless | so there's a more sophisticated error checker you can create | 17:17 |
| lifeless | have you seen Release It! - the book ? | 17:17 |
| bigjools | it uses failure.getTraceback() | 17:17 |
| bigjools | I have not | 17:17 |
| lifeless | ok, and failure.getTraceback() is neutered for some reason ? | 17:17 |
| bigjools | 2010-11-23 16:39:28+0000 [Uninitialized] Traceback (most recent call last): | 17:18 |
| bigjools | 2010-11-23 16:39:28+0000 [Uninitialized] Failure: twisted.internet.error.TCPTime | 17:18 |
| bigjools | dOutError: TCP connection timed out: 110: Connection timed out. | 17:18 |
| bigjools | that's it | 17:18 |
| lifeless | how frustrating | 17:18 |
| bigjools | somewhat | 17:19 |
| bigjools | it's thrown when we get a errno.ETIMEDOUT | 17:20 |
| lifeless | righto, the pattern is Circuit Breaker | 17:20 |
| lifeless | its kindof the ultimate variant of what you've implemented | 17:20 |
| lifeless | its not relevant right now | 17:21 |
| lifeless | but I'm going to explain why I was thinking of it | 17:21 |
| === salgado-lunch is now known as salgado | ||
| lifeless | it has the idea of the thing it measures being ok, being in trouble, and being dead. | 17:21 |
| lifeless | *if* we got the full traceback and were only logging one line | 17:22 |
| lifeless | I was going to suggest logging the full line on the transition from in-trouble->dead | 17:22 |
| lifeless | s/full line/full thing/ | 17:22 |
| lifeless | however, thats not the issue we're facing, so it was just a short side discussion. | 17:22 |
| bigjools | that's what it's trying to do, effectively | 17:22 |
| lifeless | yah | 17:24 |
| === beuno-lunch is now known as beuno | ||
| lifeless | so | 17:24 |
| lifeless | we're getting a short tb | 17:24 |
| lifeless | IIRC Failure does that when the error is thrown locally and the frame above is the reactor | 17:25 |
| bigjools | there's only 2 places the Twisted code itself can throw that error | 17:25 |
| lifeless | so could we be dealing with non twisted code doing a regular socket call and throwing, in a callback from some twisted code. | 17:25 |
| bigjools | twisted/internet/tcp.py | 17:25 |
| === deryck[lunch] is now known as deryck | ||
| bigjools | doConnect() calls self.failIfNotConnected(error.getConnectError .... | 17:26 |
| bigjools | what I don't get is that none of that seems to be asynchronous | 17:26 |
| === benji is now known as benji-lunch | ||
| bigjools | lifeless: bear in mind that this code works absolutely flawlessly on dogfood | 17:27 |
| bigjools | even with more builders added | 17:27 |
| lifeless | I think- I'd need to check some lower level code - but I think that that is: | 17:29 |
| lifeless | 'read the last error from the socket' | 17:29 |
| lifeless | failIfNotConnected looks like a plausible issue | 17:30 |
| lifeless | so getConnectError generates a stackless exception object | 17:32 |
| lifeless | and twisted.python.failure when given a regular Exception with no frame data bails | 17:36 |
| lifeless | elif not isinstance(self.value, Failure): | 17:36 |
| lifeless | # we don't do frame introspection since it's expensive, | 17:36 |
| lifeless | # and if we were passed a plain exception with no | 17:36 |
| lifeless | # traceback, it's not useful anyway | 17:36 |
| lifeless | f = stackOffset = None | 17:36 |
| lifeless | thats why you're not getting anything useful. | 17:36 |
| lifeless | *I think* | 17:37 |
| lifeless | jml: ^ plausible? | 17:37 |
| lifeless | the failure.Failure construction call is passing in a simple Exception object | 17:38 |
| lifeless | that has no traceback and thus doesn't get one | 17:39 |
| lifeless | if we passed *no* exception in, sys.exc_inf would have been called, which gets a tb object. | 17:39 |
| bigjools | the err is an errno | 17:39 |
| bigjools | right | 17:40 |
| lifeless | I'm surprised that at the claim that getting a stack from scratch is more expensive than was sys.exc_info does (when the exception is triggered). But perhaps it is. | 17:40 |
| bigjools | it might be worth a quick cowboy | 17:40 |
| lifeless | (I mean, I know it has overhead and I avoid doing it casually myself) | 17:40 |
| lifeless | but this function does one and not the other which is pretty surprising to me | 17:41 |
| bigjools | what I need to work out is why we get ETIMEDOUT so quickly | 17:42 |
| bigjools | TCP sockets take minutes to time out (hours?) by default | 17:42 |
| lifeless | depends on the state | 17:43 |
| lifeless | during connection its 30 seconds (IIRC) | 17:43 |
| bigjools | I've never heard of that | 17:43 |
| bigjools | actually, hmmn | 17:43 |
| persia | 30 seconds of retry for a packet that won't send, but much more for just an open socket without traffic. | 17:44 |
| lifeless | also 30 seconds or so for a SYN that isn't acked | 17:44 |
| bigjools | right | 17:44 |
| lifeless | but blackholes | 17:44 |
| lifeless | bigjools: http://www.faqs.org/docs/iptables/tcpconnections.html | 17:45 |
| lifeless | Table 4-2. Internal states | 17:45 |
| lifeless | bah | 17:45 |
| lifeless | thats the firewall side | 17:46 |
| bigjools | SYN_SENT2 minutes | 17:46 |
| lifeless | which has timeouts set *above* what tcp needs | 17:46 |
| lifeless | anyhow, default syn timeout is 30 seconds | 17:47 |
| jml | lifeless: yeah, the stack analysis is correct. see #twisted for some follow-up discussion. | 17:48 |
| * jml gone again | 17:48 | |
| lifeless | jml: I see discussion on the socket, not on the poor stack info | 17:49 |
| lifeless | jml: thanks though | 17:49 |
| bigjools | lifeless, did yo usee the comment " it's very easy to run out of listen queue in a python server with many short-lived connections" | 17:52 |
| === al-maisan is now known as almaisan-away | ||
| lifeless | bigjools: do you make multiple outstanding SYN attempts to a asingle build slave | 17:54 |
| lifeless | s/you/we/ | 17:54 |
| bigjools | lifeless: the code has no knowledge of SYN attempts, it's using xmlrpc.Proxy | 17:56 |
| lifeless | ok | 17:56 |
| bigjools | but it will try and connect every5 seconds | 17:56 |
| lifeless | does it make multiple outstanding xmlrpc calls ? | 17:56 |
| bigjools | actually, 15, I changed it | 17:56 |
| bigjools | no | 17:56 |
| lifeless | then that comment is irrelevant | 17:56 |
| lifeless | if we made multiple requests at once | 17:57 |
| lifeless | then we could exceed the default listen size (8) | 17:57 |
| bigjools | there must be some other difference between dogfood and production | 17:59 |
| bigjools | I have not seen this problem *a single time* on DF | 17:59 |
| bigjools | and now I have to go to dinner | 17:59 |
| bigjools | I'll catch up with you again later | 18:00 |
| sinzui | flacoste, mumble | 18:34 |
| === maxb_ is now known as maxb | ||
| === benji-lunch is now known as benji | ||
| sinzui | flacoste, https://bugs.launchpad.net/launchpad-registry/+bug/621778 | 19:21 |
| _mup_ | Bug #621778: Register project from source package should include homepage URL <qa-ok> <Launchpad Registry:Triaged by jelmer> <https://launchpad.net/bugs/621778> | 19:21 |
| lifeless | thumper: how was pyconz | 19:36 |
| lifeless | finally | 19:45 |
| lifeless | we're getting back into shape | 19:45 |
| lifeless | Time Out Counts by Page ID | 19:45 |
| lifeless | HardSoftPage ID | 19:45 |
| lifeless | 773783Archive:+index | 19:45 |
| lifeless | 54162BugTask:+index | 19:45 |
| lifeless | 26298Distribution:+bugs | 19:45 |
| lifeless | 2599ProjectGroupSet:CollectionResource:#project_groups | 19:45 |
| lifeless | 1746DistroSeries:+queue | 19:45 |
| lifeless | 1510Person:+commentedbugs | 19:45 |
| lifeless | 12256POFile:+translate | 19:45 |
| lifeless | 92Person:+bugs | 19:45 |
| lifeless | 64DistributionSourcePackage:+changelog | 19:45 |
| lifeless | 61Person:+related-software | 19:45 |
| lifeless | -> hospital for allergy vaccination, back in ~3 hours. | 20:00 |
| lifeless | on mobile if needed | 20:00 |
| === matsubara is now known as matsubara-afk | ||
| === salgado is now known as salgado-afk | ||
| flacoste | thumper: field.setTaggedValue('has_structured_doc', True) | 21:45 |
| flacoste | widget.context.queryTaggedValue('has_structured_docstring') | 21:46 |
| flacoste | field = Attribute('The <a>link</a>') | 21:48 |
| flacoste | field.setTaggedValue('has_structured_doc', True) | 21:48 |
| flacoste | field = has_structured_doc(Attribute(...) | 21:49 |
| flacoste | field = exported(has_structured_doc(Attribute(...))) | 21:49 |
| lifeless | bacj | 23:08 |
| poolie | hi lifeless, flacoste | 23:18 |
| lifeless | hiya | 23:18 |
| lifeless | flacoste: quesetion for you | 23:20 |
| lifeless | flacoste: if you're still around | 23:20 |
| flacoste | lifeless: will be gone after i hit send on that email | 23:20 |
| flacoste | but shoot | 23:20 |
| flacoste | hi poolie | 23:20 |
| poolie | lifeless: i think jam would like someone (maybe not you) to state that creating bin/bzr is/isn't the most tasteful way to do it | 23:21 |
| flacoste | poolie: i say it is | 23:21 |
| poolie | jam: ^^ | 23:21 |
| flacoste | poolie, jam: try talking to gary tomorrow for help | 23:21 |
| lifeless | flacoste: rt 41361 - I mailed you | 23:21 |
| flacoste | lifeless: yes, i saw that | 23:21 |
| poolie | that's what i said too :) | 23:21 |
| lifeless | jam: creating bin/py is the most tastesful way to do it. | 23:22 |
| lifeless | jam: I just don't know the machinery in that stack yet. | 23:22 |
| lifeless | flacoste: did I make sense? I didn't see a reply. | 23:22 |
| flacoste | lifeless: i understand it, i'll have to see how it fits resource wise | 23:22 |
| flacoste | since it needs some amount of coordination on our side | 23:23 |
| lifeless | flacoste: very small :) - I've done the heavy lifting. | 23:23 |
| flacoste | well | 23:23 |
| lifeless | flacoste: anyhow, shoo. | 23:23 |
| flacoste | it requires doing measurements | 23:23 |
| flacoste | and saying +1 / -1 | 23:23 |
| flacoste | that's not very small in my book< | 23:23 |
| lifeless | flacoste: that doesn't need coordination | 23:23 |
| lifeless | flacoste: that can be done weeks or months later, if needed. | 23:24 |
| flacoste | it needs somebody on the lp side to work with the losa | 23:24 |
| flacoste | hmm, ok | 23:24 |
| flacoste | but | 23:24 |
| flacoste | actually | 23:24 |
| flacoste | is it that important to do just one | 23:24 |
| flacoste | if we don'T assess and then deploy it across the board? | 23:24 |
| lifeless | its important to get some headroom | 23:25 |
| lifeless | we can't do this on all the servers [yet] - not enough ram | 23:25 |
| thumper | flacoste: why google docs??? | 23:25 |
| thumper | my normal gmail login can't see it | 23:25 |
| lifeless | if this works we can probably get headroom without more hardware. | 23:25 |
| flacoste | thumper: i'm sending a normal email | 23:25 |
| * thumper throws hands up | 23:25 | |
| thumper | flacoste: thanks | 23:25 |
| lifeless | thumper: a logout/login can help. | 23:26 |
| lifeless | thumper: also turn on MultiSession | 23:26 |
| thumper | I don't know what multi session is | 23:26 |
| lifeless | thumper: there's a link in my facebook feed :) | 23:26 |
| lifeless | thumper: when I whinged about logins obliterating each other on google apps/gmail | 23:27 |
| flacoste | lifeless: but my gut feeling is that completing RFWATD takes precedence | 23:27 |
| lifeless | flacoste: thats crucial as well. | 23:27 |
| lifeless | :) | 23:27 |
| lifeless | they're all critical! :> | 23:27 |
| lifeless | flacoste: seriously, gnight. | 23:28 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!