[00:26] wgrant: no, they aren't. [00:26] he was going to. [00:26] * wgrant blames email. === Ursinha-afk is now known as Ursinha === matsubara is now known as matsubara-afk [01:54] wgrant, StevenK: ping [01:54] flacoste: when you get up [01:54] https://chinstrap.canonical.com/~stub/splitit/4vs5/diff_shipit-20090810T_212109_vs_204654/ [01:54] do either of you know where build queue entries are kept? [01:55] BuildQueue, you mean? [01:55] flacoste: the request response time degrades terribly as load goes on in that benchmark - we'd want it to stay flat [01:56] wgrant: perhaps [01:56] flacoste: according to that , we'd be able to run at 30-40 concurrent users max on that config [01:56] thumper: What are you looking to do, exactly? [01:56] wgrant: I'm chasing down some weird old buildfarmjobs [01:56] wgrant: there are some entries there that aren't being dispatched [01:56] wgrant: and I'm wondering why [01:56] thumper: These aren't the same ones that don't have a specific_job? [01:57] wgrant: there are 15 old recipe buildfarmjob rows, 12 of those don't have a sourcepackagerecipebuild, 3 do [01:57] I'm wondering what happened to those 3 [01:58] thumper: SourcePackageRecipeBuildJob links SPRBs to Jobs. BuildQueue.job then references that Job. [01:58] So there are three rows that you need to look for for each of those three builds. [01:59] so to go from the buildfarmjob to the buildqueue you need to go through 5 tables? [01:59] that sounds freaking insane [01:59] The links that I just referenced are all scheduled for demolition. [02:00] BuildFarmJob is going to replace all of them. [02:00] wgrant: so BuildFarmJob is the new goodness? [02:00] thumper: That's right. [02:01] wgrant: and sourcepackagerecipebuild is staying? [02:01] Now Translations is on it, I just need to find a free weekend and delete the rest. [02:01] Right. [02:01] SourcePackageRecipeBuild*Job* is dying. [02:01] ok [02:02] but builds are being dispatched through the buildjob right now? [02:03] Yes. [02:03] but not for much longer? [02:03] Correct. Soon only BuildFarmJob, PackageBuild and SourcePackageRecipeBuild will be involved. [02:03] ok [02:03] The BuildQueue and SourcePackageRecipeBuildJob tables will vanish, and everyone will be a lot less confused. [02:03] and no links to the Job table at all? [02:03] Once we have everything sitting well-abstracted on top of it, BuildFarmJob will probably migrate to storing most of its data in a Job. [02:03] cool [02:03] But that's not a significant change, so hasn't been thought about too much yet. [02:03] thanks for the help [02:04] I decided that having BFJ store things in a Job from a start would be even more confusing: it would mean having two Jobs for every build farm task during the migration period. [02:06] so... [02:06] these other three rows have no sourcepackagerecipebuildjob links [02:06] You're not checking on staging, are you? [02:06] hence not being built? [02:07] I am right now [02:07] but I can fling the query at spm [02:07] The staging restore dequeues builds. I don't know if it deletes SPRBJs too. [02:07] But it definitely deletes the BuildQueues. [02:12] so generally speaking i feel i should add fixtures rather than add to the LaunchpadObjectFactory [02:12] is that true? [02:12] poolie: who said what about the factory? [02:14] poolie: can you give me an example? [02:14] sorry, my recollection of it is vague [02:14] i thought that abentley said it was quasi-deprecated? [02:14] or not a good pattern? [02:14] Sampledata is deprecated. [02:14] the example is if i want a thing to establish feature rules for testing [02:14] The factory is very much not. [02:15] wgrant: in the end, the BuildQueue rows are missing for the respective jobs [02:15] thumper: Do they have SPRBJs? [02:15] wgrant: yes [02:15] wgrant: and Jobs [02:16] thumper: There's nothing special about the three? [02:16] wgrant: not that I can fathom without actually looking :) [02:16] wgrant: they are from july [02:16] It could just be a botched deletion, I suppose. [02:17] yeah, I suppose [02:17] If it hasn't happened in three months... [02:58] are daily recipe build jobs being created? [03:32] https://bugs.edge.launchpad.net/launchpad/+bug/649500 is amusing [03:33] I think I saw a bug about that a while ago. [03:33] The issue has certainly been present forever. [03:34] can i have just features/browser.py, or should it be a submodule? [03:34] ah i see there are some precedents for a plain file [03:34] Bug #640435 [03:34] <_mup_> Bug #640435: branch name conflicts with zope.ManageContent [03:34] * mwhudson wtfs at poolie's bug [03:35] i guess it's inherited from some builtin zope thing [03:35] is it a cms or a framework? who can tell [03:36] ah yeah [03:36] Well, the ZTK was meant to fix that duality. [03:36] I think it has. [03:37] j'accuse zope.app.publisher [03:38] yep [03:38] Er. [03:39] Why do we use that? [03:39] \o/ [03:39] * rockstar conquers buildout [03:39] specifically BrowserSkinsVocabulary [03:39] I would have thought all our needs would be in zope.publisher. [03:39] it doesn't seem to add anything we do want [03:40] lib/canonical/launchpad/doc/zcmldirectives.txt:# , [03:40] WTF? [03:40] I get the feeling that I don't want to look at that file. [03:40] well hehy, if we can fix a bug by dropping a dependency [03:41] We still import a whole lot of stuff from it. [03:41] But I'm fairly sure it was all moved to other packages ages ago. [03:42] Oh look, zope.app.publisher imports them from zope.browserpage. [03:42] wgrant: certainly looks like it [03:42] Hm. [03:42] But we still need it for XML-RPC :( [03:45] it's probably not very hard to write a zcml directive that unregisters the offending view [03:46] I'd personally prefer to see if we can stop needing IMethodPublisher, since that's our remaining dependency on zope.app.publisher. There must be a replacement... [03:47] it [03:47] seems pretty tiny [03:47] It really is. [03:48] Hm. Looks like z.a.p might provide the xmlrpc:view ZCML directive. [03:48] So it may be harder to excise. [03:48] Why is that in there, I wonder... [03:50] Ah, the ZTK steering committee appears to have just not yet decided where to put it. [03:50] Grar. [03:52] Hmm. [03:52] What happens if we just remove the zope.app.publisher ZCML inclusion? [03:52] And import just the zope.app.publisher.xmlrpc [03:52] * wgrant tries. [03:54] That seems to have worked. [03:56] i wonder if there's some way of finding all the places where we import from deprecated locations [03:56] Given the recent ZTK reshuffles, that's probably most of our imports... [03:57] All the xmlrpc tests are happy with my fix, and the webapp seems to work. [03:58] I'm not sure how, though. [03:58] Something else must be pulling in the zope.browserpage and co. [04:18] lifeless: got any ideas about https://bugs.edge.launchpad.net/launchpad-code/+bug/537116 ? I don't know where to start looking. [04:18] <_mup_> Bug #537116: bzr branch from launchpad disconnected after exactly one hour === thumper changed the topic of #launchpad-dev to: Launchpad Development Channel | Performance Tuesday ! | Week 3 of 10.10 | PQM is open for business | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting [04:24] thumper: his firewall perhaps ? [04:24] * lifeless shoots first, asks questions later [04:24] we could try and reproduce [04:25] I suspect something at his end though, because e.g. pulls of LP can be an hour and work [04:42] there's no timeout on our end? [04:43] it's plausible it's his firewall [04:43] a bit strange if it kills an actually active socket though [04:44] spm: do we have anything that would kill a bzr serve process / ssh connection after 1 hour ? [04:44] I can think of several possibilities, but don't *know* which is likely. [04:45] i'd be asking for a snoop that shows the termination. that may be a clue. [04:45] we have code in launchpad that's supposed to kill _idle_ connections after an houe [04:45] *hour [04:45] as far as whether it's killed by a reset or socket close [04:46] mwhudson: where is that code? [04:46] poolie: not sure now since jml shuffled this area [04:46] * mwhudson pokes [04:46] if it was an econnreset it would be more likely to be a router intervention, i think [04:46] but it's just an early eof, which could be anything [04:47] perhaps is slightly more likely to be app code than networking [04:47] poolie: lp.services.sshserver.service [04:47] poolie: line 150-ish [04:49] oh right [04:49] well he's definitely being cut off by the idle_timeout [04:49] but what's happening is that the branching runs for an hour [04:49] poolie: line 150-ish <== was that from memory? or did you look it up? ;-) [04:49] then the connection goes idle [04:49] then an hour after that he's disconnected [04:49] spm: i looked it up :-) [04:50] :-) [04:50] poolie: there is also a bug in conch's ssh key renegotiation [04:50] that usually kicks in after a gigabyte of traffic [04:50] he doesn't seem to be close to that though [04:52] aside from anything else we should upgrade the origin branch [04:56] well, it would be an incredible coincidence for that to happen after an hour [04:57] but perhaps there's a time-based renegotiation too? [04:57] do you have a bug number for that? [04:58] it wouldn't be some reverse speed of light problem maybe? http://www.ibiblio.org/harris/500milemail.html [04:59] poolie: yes, but i can't find it [05:03] poolie: I'm contacting the project owner [05:03] web2py owner that is [05:07] spm if we have users more than one light-hour away, uh.. [05:07] i'll be impressed [05:07] you never know...... [05:08] about 1e9 km [05:09] tcp over spaceship would be painful [05:09] talk about slow start [05:10] lifeless: you've not seen that rfc? it's an update on 1149 [05:10] carrier pigeon? [05:10] http://twitter.com/#!/colmmacuait/status/25352220323 [05:11] nice [05:11] lifeless: indeed [05:11] bwhahaha [05:11] so in a bit of poking around i can't see anything in conch to do with automatically renegotiating keys, based on either traffic or time [05:11] it may well be there but i don't see it [05:11] poolie: it's client side [05:12] spm: heh, that was a good one about 500 miles [05:13] nigelb: *very* old favourite. The Story Of Mel, is another [05:13] spm: Story of Mel? [05:13] http://www.pbm.com/~lindahl/mel.html [05:14] mwhudson: and not implemented by conch as a client? [05:14] poolie: probably not [05:14] mwhudson: but ok, i can see how openssh might ask for a new key after a few gb and then that could cause trouble with the server === Ursinha is now known as Ursinha-afk [05:14] but that's almost certainly not what we're seeing here [05:14] this is infuriating, i spent quite a while looking at this and writing it up and now i can't find it [05:14] poolie: right [05:14] mwhudson: i'm trying to branch from there [05:15] poolie: i think openssh's default is 1 gig [05:15] you can change it [05:15] my connection is faster so it may just complete [05:15] says "depending on the cypher 1-4GB" [05:16] spm I watched "Colossus: The Forbin Project" last night [05:16] had to bite my tongue a lot [05:16] "ffs it never works first time" [05:18] poolie: I've never heard of that one! I shall have to chase! [05:18] visually beautiful === almaisan-away is now known as al-maisan === Ursinha-afk is now known as Ursinha [07:38] lifeless: https://bugs.launchpad.net/bugs/637507 if you haven't seen it. I don't understand the low level stuff or the risks, but sounds like something we might want to do across the board. [07:38] <_mup_> Bug #637507: Uploads with extra data in the .tar.gz rejected unnecessarily

[07:38] stub: hey [07:38] stub: see the cassandra mail? [07:39] stub: sigpipe handling; yes, it should really be reset by subprocess automatically. [07:39] lifeless: yes, and pfibiger pinged me this morning. [07:40] stub: a wrapper for it would be useful [07:40] lifeless: I have to think about the timing for the cassandra - my lease on a new house starts the same day. [07:41] meep [07:41] lifeless: knowing the location would be helpful - short haul might be ok [07:41] us I believe [07:41] lifeless: I was told 'up to mariana' [07:41] ah, interesting [07:42] u1 is us centric though, so probably dullas again [07:42] would taipei count as short haul ? [07:42] yes [07:42] I suggested Taipei. 'We should all get to Taipei' I recall from a keynote. [07:43] Only problem with Taipei is I would want to take a holiday there and I can't ;) [07:43] heh [08:24] Why is Dallas so popular? [08:31] cheap, reasonably central [08:31] (I guess, I have no knowledge) [08:33] wgrant: hey, do you happen to know the name of the signal raised in +filebug w/apport ? [08:33] lifeless: Signal? [08:42] event [08:42] thingy dookhicky [08:42] lifeless: To do what? [08:42] I don't know of a special event. [08:53] good morning [08:57] Hi there. I've started getting 'Errors in work item definitions' from 'Launchpad work item tracker ' [08:57] emails. How can I stop them? [08:57] michaelh1: thats not an lp service, for all that its using the launchpad.net domain [08:58] last i heard Martin Pitt was running it [08:58] bigjools: hi; CPing your thing ? [08:58] [thats code for 'had a good weekend?'] [08:58] Right, I'll track pitti down... [09:00] moin [09:03] wgrant: this is what I was meaning [09:03] http://pastebin.com/SHStVhUv [09:04] though, that may not be enough [09:04] I think I need to get a profile [09:04] (can apport be pointed at staging?) [09:05] do i need an __all__ even in tests? [09:06] poolie: Only if they export something [09:07] thanks [09:10] Mornin' [09:10] hi there [09:14] lifeless: APPORT_STAGING=1 ubuntu-bug linux [09:14] IIRC [09:16] the main reason we need __all__ is the import fascist requires that imports be in it [09:16] this is a bit silly [09:16] because you can do module._thing [09:16] you mean it doesn't really guard against using private interfaces? [09:17] right [09:17] it doesn't seem to have any functional benefit [09:18] I think it's a handy check. [09:18] If it works. [09:18] But I don't think it does. [09:18] it hasn't whinged at me much lately. [09:18] Not being very fascist at all. [09:18] I question the value now that security is on the model [09:19] s/ now/ particularly now/ [09:19] It makes some attempts at preserving layering. [09:20] assuming people actually respond by changing the layering, not just updating the paperwork [09:21] wgrant: with the assumption that the layering is useful; the primary differences between view and model code are: [09:21] - some model code is db backed [09:21] - model code wasn't allowed to cache for a while [09:21] - model classes get interfaces (high overhead) and security (useful where it matters) [09:22] re bug 646563, is it really true that login(something); getUserBrowser() doesn't give you a browser as that user? [09:22] <_mup_> Bug #646563: want a way to make a TestBrowser logged in as a particular user [09:22] the first item is a performance problem, the second isn't relevant anymore and the third adds a good 15+ percent to the code base, but *mostly* is just boilerplate. [09:22] let's not forget that the import fascist also prevents us from using storm sorting in view code [09:22] jml: yes, an import 'benefit', that one. [09:23] jml: if you wanted to remove it, I'd be fine with that. [09:23] lifeless: landscape has an importguardian that's, well, better [09:23] jml: I remain fundamentally unconvinced that this is appropriate for python, at all. [09:23] lifeless: last time I tried this, I went to split that out into lazr.importguardian, but got blocked on new lazr packages being very hard to make [09:23] poolie: yes, its true. [09:24] jml: why shouldn't we just remove it? [09:24] What does the guardian do? [09:24] lifeless: don't know, basically [09:25] wgrant: I can't remember the details beyond "like the fascist, but nicer; let's you import for sorting" [09:25] Why were they both created in the first place? [09:25] ahh, now _that_ is a historical question [09:25] so the answer is clearly "for historical reasons" [09:25] Heh. [09:25] i have code to add getUserBrowserAsTeamMember in https://code.edge.launchpad.net/~mbp/launchpad/flags-gui/+merge/36415 [09:26] i might just move that to BrowserTestCase as part of this same mp? [09:26] poolie: Is that really worth its own method? [09:26] I suspect it was to stop programmers from doing bad things like circumventing the interface/security code [09:26] wgrant: how would you write a test that needs a logged in user? [09:26] (as opposed to getUserBrowser(getTeamMember(blah))) [09:26] frankly, that sort of thing, unless there is a really big win, just annoys me [09:27] poolie: [09:27] wgrant: that doesn't seem to work [09:27] lifeless: well, yes :) [09:27] user = self.factory.makePerson(password='test') [09:27] browser = self.getUserBrowser(user=user) [09:27] right, then add them to those teams [09:27] it's only 5 lines but assuming it doesn't exist already, it seems reusable [09:27] I think its a bit of a kludge [09:28] how about this [09:28] getUserBrowser(user=user) resets the users password if its wrong [09:28] or [09:28] how about we make the default password in makePerson be 'test' [09:28] uhm [09:28] I'm not against a convenience helper gluing stuff together [09:28] anyway, I must ablute. back later. [09:28] mm, or i could add a teams=None parameter to getUserBrowser [09:28] I think I'm mainly reacting to the fact that one is needed here. [09:29] poolie: I don't understand why team membership and browser instance creation are coupled. [09:29] they seem like vastly different concerns. [09:29] if you did this: [09:29] getUserBrowser creates a new person [09:29] - change makePerson to use 'test' as the default password [09:30] then you can do the following: [09:31] then gUB will 'just work' [09:31] and if you want a team relationship you can make that and pass in a user (and again it will just work) [09:32] i'm reluctant to change something called by so many tests as makePerson [09:33] and yet that is what makes it valuable to change [09:33] losa ping; I'd like to request profiling on staging please [09:34] lifeless: is it urgent - working on something just at the moment [09:35] mthaddon: I can't claim urgent, but its 2130 for me, if I can get this before I head to sleep I may be able to confirm/deny a theory for the +filebug timeouts. [09:35] and kick it over to deryck for further examination [09:36] can you file an RT and I'll get to it a little bit later on (if you can include the info that's needed to hand over to deryck)? [09:36] another option would be to stash the unhashed password on the object [09:36] mthaddon: no, the info is me analysing kcachegrind [09:36] mthaddon: which is what the profile would give me; if you get a timeslice ping me [09:37] mthaddon: if you don't ping, thats fine, I'll do with someone tomorrow morning. [09:37] lifeless: k, will do - how long of a timeslice would you need? [09:37] lifeless: Do we not have a URL trigger for that yet? [09:37] mthaddon: you edit one config file and restart the appserver; I go play until I have the data; then you revert the config file and restart, and trigger your rsync that copies the profile files to sodium. [09:38] wgrant: we do, but because its not secured it requires a config change to enable it. [09:38] lifeless: ... ah. Useful. [09:38] wgrant: next step is a featureflag to enable it, and a group scoped to 'launchpad' [09:40] sorry, a scope, which looks up the ~launchpad group. [09:41] mthaddon: http://pastebin.com/yTp0TL1D is the change; I'm sure this is in LOSA docs already [10:08] lifeless: diff applied and staging restarted [10:11] thank you [10:13] mthaddon: thanks; revert it anytime you like; and please kick off the rsync of the profiles (AFAIK its not cronned; maybe it is but just very infrequent) [10:13] mthaddon: no rush [10:23] lifeless: realize it's dead late but if you sometime want to do a meta-review on https://code.edge.launchpad.net/~mbp/launchpad/flags-gui/+merge/36415 [10:23] that could be good [10:23] especially as to whether the ui is adequate to start with [10:25] we could do that thursday [10:33] poolie: it sounds fine to me [10:33] mthaddon: when things are not on fire, perhaps you would like to comment on the devops-usability of that mp [10:33] s//of the feature proposed in that mp [10:33] thansk [10:33] i'm a bit impatient to get it done [10:33] poolie: its better than sql, so I'd just land it [10:33] yeah :) [10:33] i appreciate the reviews though [10:48] bigjools: so, are you going to CP 11566 and 11579 (IIRC) today? [10:48] lifeless: jelmer is preparing a CP, yes [10:49] not sure of the revnos [10:51] great [10:51] we noted that one of the bugs is not qa'd [10:51] which? [10:51] bug 128259 [10:51] <_mup_> Bug #128259: Buildmaster should not call "uploader" script for processing incoming binaries

[10:52] * bigjools f ixes that [10:55] pylint blows [10:56] yes [10:57] * bigjools considers fixing the lint make target [10:57] if I cared a little bit more, I'd write a Python formatting checker/fixer that was as fast as pyflakes [10:57] it really hates our new import style [10:57] and then nuke pylint with great nuking [10:58] we should just get rid of pylint [10:58] it's worse than useless [10:58] no arguments here. [10:59] the reason I want a formatting checker/fixer is I'm sick of thinking about formatting. [10:59] but that doesn't block removing pylint [10:59] * bigjools blinks [11:00] urgh, a shell script [11:36] I'm closing my mail & IRC clients for a bit to get some work done. Call my phone if you need me. === al-maisan is now known as almaisan-away [12:03] Morning, all. === didrocks1 is now known as didrocks [12:29] bigjools: The publisher will no longer die if there's an unconfigured series, will it? [12:29] wgrant: no idea [12:30] Yeah, it looks like it was fixed a couple of months ago. [12:31] (#ubuntu-devel just mentioned that they couldn't target specs to natty yet) [12:40] wgrant: That was one of the booby trap issues I fixed last release IIRC [12:40] bug 55288 [12:40] <_mup_> Bug #55288: publisher gets very unhappy about unknown distroreleases

[12:41] \o/ [12:41] That's the one. [12:42] Also, I hate ISPs. [12:42] My international routing has just collapsed for the fourth time this evening. [12:42]