[00:02] wgrant: should i kill it in ec2? [00:15] Let's see what version of python-apt is on pigeonpea/pilinut/cocoplum/germanium/cesium. [00:15] LOSAs: ^^ [00:16] wgrant: one sec [00:18] wgrant: 0.7.94.2ubuntu [00:18] There are three characters missing there. [00:18] And they are the critical ones :) [00:19] 0.7.94.2ubuntu6.2, does that sound right? [00:19] That sounds right. [00:19] But also old :( [00:19] poolie: ^^ we need 6.4 [00:24] ok [00:24] hloeung: do those machines not get l-updates automatically? [00:25] well, i realize approximately no machines upgrade really automatically [00:25] but would it be hard to bring it in? is it seen as being available? [00:34] poolie: yeah, it seems to be available. [00:34] i killed the ec2 land run and i'm running just a test [00:35] thanks haw; how would we go about getting it upgraded? [00:35] a request on the lp production status page? [00:36] file an RT about it [00:36] and we'll do it for you [00:45] lifeless: should i/we do this ^^? [00:50] poolie: I'd rather someone involved with the bug do it [00:50] the side effects of an uncoordinated change can be severe [00:50] mm [00:51] who would that be? jelmer? [00:52] jelmer, wgrant, bigjools, gavin, stevenk, jtv or colin - offhand those folk should all have the experience to make reasonable predictions about side effects [00:52] cool [00:53] i'm pushing it because it has been mentioned as a udd blocker [00:54] so basically the branch can't land until we've separately done the upgrades everywhere [00:54] and that needs teseting and QA in its own right. [00:54] e.g. start with buildbot / *staging, dogfood, check its all good, then prod, then and only then land the branch. [00:55] yep, thanks [01:01] poolie: A UDD blocker!? [01:05] well, not a blocker for packaging branches, but a thing people complained about in our meeting [01:06] Interesting. [01:07] 'cause it's not really related at all... [01:11] poolie: what trouble was it causing them ? [01:13] i believe it was that they wanted to sync tar.xz packages from debian [01:13] ? [01:13] the comment was basically just "can we have that xz bug fixed?" [01:14] heh [01:14] well, I'm glad you have an interest in it [01:14] it will be good to have it fixed [01:15] .. and i just hate to see thing stalled, at least without a "this is waiting on" comment [01:15] lifeless: do you want to read https://code.launchpad.net/~mbp/launchpad/mail-scope/+merge/60281 again? [01:15] it has a +1 [01:15] sure, i can do that [01:29] Has anyone seen http://paste.ubuntu.com/655990/ before? [01:31] i think i've seen something like that, where _testMethodName is missing [01:31] as the name implies its failing to load and then things are not sufficiently set up to report the faliure [01:31] i'd suspect a syntax error or similar [01:31] or an import erorr [01:37] pyflakes is happy, which is the odd thing [01:42] i would probably bisect backwards to find something that works [01:43] If I comment out a class variable "foo = PillarName.projectgroup == None" it works [01:47] ah, well maybe that's stopping it loading [01:48] perhaps projectgroup is not defined at the time that initializer runs? [01:48] Hmmmm, maybe it isn't projectgroup [01:50] lifeless: shall i wait for you to review the mail_header patch or send it now? [01:52] Rargh! [01:52] PillarName has product and project [01:52] Which one is projectgroup, then? [01:53] project [01:53] Product = Project, Project = ProjectGroup [01:54] poolie: I have reviewed it now; I had to pop out to the vet before. [01:54] np, thanks [01:54] poolie: I think its fine as is but would be smaller if you derive from Fixture [01:55] wgrant: That's ... obvious :-/ [01:55] lifeless: oh, interesting idea, even though it's not used only in testing? [01:56] fixtures isn't just for testing [01:56] never was [01:59] ok [03:01] wallyworld: hi [03:01] o/ nigelb [03:01] hey poolie! [03:01] nigelb: hello [03:01] how's the branch going? [03:01] wallyworld: I was working on solving that bug last weekend and ran into doubts. I emailed you :) [03:02] ..and the branch is here -> https://code.launchpad.net/~nigelbabu/launchpad/4595-upgrade-bug-linking [03:02] nigelb: really? i don't recall seeing the email [03:02] let me check my inbox [03:02] :) [03:05] nigelb: it was in my trash - i got a whole lot of code import email which escaped my filter arrive in my inbox and yours was right in the middle and accidentally got deleted [03:05] haha [03:05] give me a few minutes to read it :-) [03:09] oh. 240 critical bugs? :( [03:09] nigelb: so i see you've added the 'bug-link' class to bug links and are grabbing those to send to the back end [03:10] Yeah. [03:10] I got that far. Now at the javascript is where I have a few doubts on how to proceed [03:11] did you have a particular question? [03:11] Yes, so in the javascript. [03:11] I'm doing harvest_links(links_to_check, 'branch-short-link', 'branch_links', 'bug-link'); [03:12] Is that the right way, just add the css for the bug along with the branches [03:12] I was wondering if they should be seprate variables [03:13] the server has one end point for processing any links extracted from the page, so it makes sense just to add a new link class to the existing variable [03:13] ie gather all links, send them to the server for processing, render the results [03:13] ah, so now I have to go to the server and add stuff to the processing bit. [03:14] on the server side you can introduce a separate class for bugs vs branch links [03:14] ah, ok [03:14] the reason it can be in the one data structure is that the server is just providing generic info (enabled, title text etc) [03:15] so on the client, there's no reason to distinguish between bug vs branch links [03:15] does that make sense? [03:15] Yes, it does :) [03:16] excellent! [03:16] I guess the next place I should be making modifications is lib/lp/app/browser/linkchecker.py [03:16] yes [03:17] or you could write some tests for the javascript side ie TDD [03:17] get that side of things sorted first [03:17] ah, yes. [03:18] there may not be yui tests for that stuff [03:19] there isn't. [03:19] the framework we have now was not around back when it was written [03:19] so I guess I'll have to write new tests [03:19] yeah, that would be good if you had the time [03:19] I'll try :) [03:20] Not sure if I can get them rright. [03:20] or at least write tests for the stuff you are adding [03:20] i can help [03:20] I'll push to the branch tonight and pke you tomorrow to see if I've done it right [03:21] nigelb: you could look at a simple existing test and use that to get started eg test_hide_comment.js [03:22] I did look at one [03:22] It looks easy enough. [03:22] also http://dev.launchpad.net/JavascriptUnitTesting [03:22] (famous last words, I know :P) [03:22] yep :-) [03:23] hehe, who wrote that wiki page? [03:23] "Day one was horrible, there was a lot of shouting, words were said that may have hurt my computer's feelings" [03:23] not sure, but that's funny :-) [03:23] yeah, heh [03:29] Interesting. [03:29] make run now takes only ~30s [03:35] Laters! Off to work. [03:44] bye [04:43] StevenK, wgrant, greetings! Continuing Q/A on the python-based publish-distro script. [04:44] jtv: While we are there, it seems that publish-distro.py itself is now causing ~1500 OOPSes a day, because each WARNING spewed by CommandSpawner from a-f stderr is an OOPS now :/ [04:47] wgrant: I imagine a lot of that simply shouldn't have been WARNING though. [04:48] That's assuming that this is mainly increased urgency & visibility for problems that were previously just as big. Or have substantial new problems cropped up? [04:48] jtv: apt-ftparchive's stderr output is... verbose. [04:49] And 65 OOPSes per publisher run is just oh so much noise [04:49] But the OutputLineHandler we give to CommandSpawner is self.log.warning [04:49] Which, in a LaunchpadCronScript, causes an OOPS. [04:49] Then that should probably just change. [04:49] Indeed. [04:49] Well. [04:49] Ideally we would tell a-f not to spam crap to stderr. [04:49] Quite. [04:49] But that sounds more difficult than s/warning/info/ [04:51] Well, be the first kid on your block to fix a thousand oopses today this week! [04:51] And not hugely more rewarding. [04:51] I'm not allowed to :) [04:51] But maybe. [04:51] Borrow abentley's certificate. [04:51] Heh. [04:52] If it's Critical and gets in the way of normal work… [04:52] (Which would equally be an excuse for me, so I could pick it up if you can't) [04:52] Critical isn't an excuse for you. [04:52] Being in scripts that you're working on is. [04:53] I was thinking of the "gets in the way of normal work" part, but yes. [04:53] Meanwhile, I need to upload test packages to the security pocket & a partner archive for further publish-ftparchive testing. [04:53] Fun. [04:53] (Have you ever used the python version?) [04:53] You can't build in security directly, so you have to copy to elsewhere. [04:53] s/to/from/ [04:53] Ah [04:53] that un-confuses me. :) [04:54] And partner? [04:54] partner you just upload with a component of partner. [04:54] (that is Section: partner/SOMESECTION) [04:54] That's in the .dsc? [04:55] Right. [04:55] Well. [04:55] And debian/control in the unbuilt [04:55] Set it in debian/control. [04:55] dpkg-buildpackage will put it in the .dsc [04:56] If you edit the .dsc directly, there will be great sins to atone for. [05:05] How should I search for truth in Storm? PillarName.active is True or PillarName.active == True ? [05:06] Although I seem to get inactive results in either case [05:10] ==True is what most of our code does [05:10] this isn't good SQL [05:10] but that should be changed in storm [05:11] I'm still concerned I'm getting inactive results when both inner-selects have PillarName.active == True [05:11] whats the full query? [05:12] lifeless: http://pastebin.ubuntu.com/656105/ [05:14] StevenK: why the ALL ? [05:15] Without the ALL it would not return the rank 100 result sometimes [05:17] the implicit distinct on a UNION is on all the columns [05:18] I suspect a bug in your _filter - it could explain both situations (the current confusion and the rank 100 missing sometimes) [05:19] lifeless: I have two subclasses, for one of them I want a filter [05:20] lifeless: If you want the actual query that the code produces, I can get that for one of the cases [05:21] StevenK: I'd like to see the actual query postgresql sees [05:21] Yes, I was going to get the query from postgres's log [05:21] StevenK: (or run with LP_SQL_DEBUG=1) [05:28] StevenK: 'is True' can't work, '== True' does. [05:31] LP_DEBUG_SQL=1 and doctests suck [05:32] Hah. System death. Just what I needed on the day I discovered I left my power supply in Europe. [05:36] lifeless: http://pastebin.ubuntu.com/656113/ [05:36] lifeless: I don't like the look of the "AND NULL" [05:37] thats your _filter; and yes, thats a little worrying ;) [05:38] lifeless: I only need to filter for one case -- I suspect it is None for the other case [05:38] a better way to construct things would be a list [05:39] And(*clauses) [or skip the and, find's default join is AND [05:51] lifeless: That looks like a better query. Sort of [05:51] .contains_string() is just shit SQL [05:52] you should be able to drop the ALL too [05:52] (unless you specifically want it) [05:53] This query is based on one that Curtis wrote for me, and that used UNION ALL [05:55] lifeless: the ALL in a UNION is usually something you want to keep if possible. [05:56] jtv: why? [05:56] Saves a filtering pass. [05:56] (And thus, potentially a sorting pass) [05:56] jtv: only if its unsorted though? [05:57] jtv: and all our vocabs are sorted [05:58] It depends on the details, which I don't know and prefer not to get into right now; what matters is the uniqueness of entire rows, and chances are that the planner can't determine statically that that's not an issue. [05:59] sure, thanks for mentioning this [05:59] (Chances are that it could in theory, but for it to try might significantly increase the run time of very fast and very frequent queries) [05:59] np [05:59] StevenK: what was the fs root for process-upload again? [06:00] /srv/launchpad.net/ubuntu-queue [06:00] thx [06:00] Can't we make this a configuration item at some point? [06:01] Not really. There are several queues in active use. [06:01] The two poppy queues (there were four, now only two), the buildd-manager queue, the sync queue, the backdoor queue. [06:02] * jtv sobs quietly [06:02] "why does it all have to be so _hard_?" [06:04] Oh for Kibo's sake… battery critical already!? [06:05] Hmm. [06:06] Is lib/canonical/launchpad/webapp/templates/launchpad-model.pt used? [06:06] It was added in r13548 and is buggy. [06:06] But grep shows nothing. [07:19] Good morning. [07:36] good morning [07:37] moin adeuring! [07:37] hi henninge! === henninge changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: henninge | Critical bugs: 240 - 0:[#######=]:256 === almaisan-away is now known as al-maisan === al-maisan is now known as almaisan-away [08:35] We are going to need a report on what db patches have been applied to (production, staging, qastaging) vs. what have landed in what branch. [08:36] Is there a smart way of seeing what files exist in the database/schema directory of lp:~launchpad-pqm/launchpad/db-devel besides checking out the tree? [08:39] loggerhead [08:39] StevenK: bzr ls [08:39] StevenK: bzr ls [08:39] Or bzr ls [08:39] argh [08:39] bigjools: FAIL [08:39] stub: bzr ls [08:39] * bigjools kicks auto-complete in the spuds [08:40] Seems to be downloading a heap, but that likely doesn't matter on the LAN. [08:40] 1.7MB and counting... [08:41] 4MB... [08:42] Is this going to download the whole tree? [08:42] hmm, it would be good for the qatagger to do this, roll it into the same report [08:42] stub: no [08:42] stub: but it will be downloading pack data, and if you are behind a broken http proxy this can get quite scattered [08:43] Ooh... this is ssh. Wonder if HTTP goes faster locally for this sort of thing? [08:43] finished now. should be fine on the lan. ta. [08:44] And using the recursive option might help [08:44] stub: or just ls database/schema [08:44] bzr ls lp:~launchpad-pqm/launchpad/db-devel was what I first did, which strangely enough got me the top level directory listing. [08:45] strange that [08:47] Is there anything stopping bzr on devpad being upgraded? Missing -d, -k on that command. [08:50] bzr ls -v lp:~launchpad-pqm/launchpad/db-devel/database/schema seems to do what I want with the old version [08:58] If course, if I want to show revisions changed as well I should probably jump in and use bzrlib rather than try to parse the output of bzr log [08:59] nothing stopping it being upgraded [08:59] stub: I'd seriously consider patching the qatagger for this [09:00] stub: the only tricky bit would be knowing what patches are live; perhaps a query page for that. [09:00] lifeless: That could be a good hook. [09:01] stub: why does the garbo merge depend on the bugsummary rollup peformance patch? [09:01] I was considering a tabular report db | db | db | branch | branch , with the db columns boolean and the branch columns a list of revisions the patch was touched in. [09:02] But maybe this is better solved with similar process like qa-foo tags? [09:02] stub: what are we solving ? [for clarity, rather than me guessing :P] [09:02] lifeless: Because the db patch not only attempted to optimize the stored procedure, it added an optional 'batchsize' argument to the rollup function [09:02] stub: ah [09:03] lifeless: juggling merges of code that depend on database patches being applied [09:03] lifeless: nothing urgent in there, so I've just added that branch to my todo list. With frequent merges, it shouldn't be a problem [09:03] stub: so, http://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-devel/revision/10832 looks like a hot patch [09:04] It is, and I will be applying it live shortly with one of deryks, but the test suite won't pass for the garbo branch without that patch in the tree. [09:04] cool [09:05] so what I've been recommending folk do is get the patch live and then land the patch to devel, and then dependent patches [09:05] this depends on us getting the latency of application down [09:05] but avoids lots of cherrypicks and hard-to-qa stuff in db-devel [09:05] perhaps this is wrong [09:06] stub: so the report; uhm; I would hope that any code which /needs/ a db patch would fail tests without it. [09:06] stub: I think a report is very useful to have. [09:06] lifeless: agreed on both [09:07] devs probably can look in a source tree easily enough for something they care about [09:07] the thing devs /cannot/ do today is determine whether a patch is live without nagging a losa / gsa/ dba [09:08] Should we just have a trivial view which returns the applied patches? [09:08] yer, the important thing is what patches are on what db (which could be done with a view in Launchpad) [09:08] snap [09:08] The report would present all that info in one place though, as well as any interesting trivial like a db patch that was modified after landing in a particular tree. [09:09] Sounds like it might belong in the deployment dashboard. [09:09] well, thats why I was suggesting qatagger. [09:09] Mmm. [09:09] unapplied db patches [09:09] I'd prefer another data provider that lpqateam.canonical.com watches. [09:09] Rather than forcing it into qa-tagger. [09:10] It doesn't seem to be a natural fit. [09:10] * lifeless shrugs [09:11] the biggest ROI seems to be on a view for what-patches-are-live-on-this-server [09:26] wgrant: can we talk about initialisation checks now? [09:31] that scared him off [09:56] bigjools: Hi. [10:32] bigjools: http://jam-bazaar.blogspot.com/2009/11/memory-debugging-with-meliae.html [11:00] henninge, hi, do you mind reviewing https://code.launchpad.net/~danilo/launchpad/bug-814580-pre-cleanup/+merge/69980? [11:01] wgrant: Hi! [11:01] wgrant: "Needs information" on https://code.launchpad.net/~wgrant/launchpad/iseriesbugtarget/+merge/69951 [11:02] danilos: sure [11:02] stub, lifeless: I've got two DB patches up for your review, one is "hot", another is "cold" (and I don't care about getting that one done "soon", since it's a table rename) [11:02] henninge, thanks [11:03] danilos: Relative packages in zcml? Are we doing that nowadays? [11:04] henninge: It needs a bug? [11:05] wgrant: well, some information about the motivation for the chage. [11:05] Indeed, but not a bug :) [11:05] wgrant: there must be some code in use (or in future use) where this will be used. [11:06] wgrant: yeah, probably, because it needs no qa. [11:06] Well, there already is one case: shouldIndentTask. I thought the utility was fairly clear, regardless of future specifics. But I shall update the MP with the real intent. [11:14] henninge, there's a bunch of that in existing .zcml, and this makes it easier to adhere to our line length limits (though, it makes it harder to grep for stuff) [11:14] henninge, I am not too attached to the formatting so if you want, I can make it complete [11:15] danilos: I was just wondering. You can leave that as it is. === Ursinha-afk is now known as Ursinha [11:17] danilos: lines 255, 261, 303: I'd do full indentations in that case. [11:18] henninge, what do you mean? 4 spaces? (this is how Emacs python-mode does it by default) [11:18] danilos: also, shouldn't test_translationpackagingjob be renamed, too? [11:18] henninge, translationpackagingjob is not renamed [11:18] henninge, ideally, it should [11:19] henninge, or well, not, because that one is for the Packaging links being introduced/removed [11:19] (it should be renamed in my follow-up branches where I add the TemplateChangeJob) [11:19] oh, I mixed that up. [11:20] danilos: never mind about the indentations, they work for me the way they are. [11:20] danilos: just needed something to complain about ... [11:20] danilos: r=me ;-) [11:24] henninge, excellent, thanks [11:32] henninge, btw, how familiar are you with zope events and testing them? (basically, I wonder about testing IObjectModifiedEvent on potemplate, and the event doesn't fire if I just set potemplate.name to something) [11:33] danilos: Zope what? ;-) [11:34] danilos: no, I am not familiar with th em [11:34] henninge, ack :) [11:34] danilos: You need to create a Snapshot and then notify() the ObjectModifiedEvent manually, unfortunately. [11:35] wgrant: really? what I find weird is that IObjectCreatedEvent and IObjectRemovedEvent are tested in the same file (albeit on a different object) and they work just fine; is this specific to the Modified event? [11:36] danilos: Those two are easy. [11:36] There are no field changes to take care of. [11:36] ObjectModifiedEvent has to diff the objects. [11:36] wgrant: true, ok, just knowing that's what I've got to do is helpful enough, it kinda felt like cheating [11:42] if you use dput to upload to a PPA that doesn't exist, when do you get the error message? [11:42] jml: An email after 0-5 minutes. [11:43] wgrant: thanks [11:43] Yes, yes, this sucks :) [11:44] And, IIRC, making it not suck requires message queues and some nebulous amount of rejiggering. [11:44] it could do that check in poppy along with the GPG check [11:44] To do it properly, yeah. [11:44] bigjools: The amount of DB access it has now is already problematic and excessive :( [11:45] We will have a message queue within months. [11:45] ??? it's tiny [11:45] for my part, I could *probably* get away with LBYL. [11:45] And the world will be a better place. [11:45] bigjools: Yes, but it's existent. [11:45] bigjools: This makes DB upgrades harder, makes it much more fragile, leaves it stuck in our tree... [11:46] henninge, there's another one I need reviewed (just need to fix this problem with event firing for a single test, rest is all good): https://code.launchpad.net/~danilo/launchpad/bug-814580/+merge/69978 === stub1 is now known as stub [11:51] danilos: don't you have any shorter ones? ;-P [11:53] thanks for review henninge :-) [11:53] gary_poster: you are welcome [11:55] henninge, heh, I do but they are DB patches :P [11:55] danilos: ah, no thanks ;-) [11:55] danilos: I'll just do this one, then ... ;-) [11:58] thanks again :) [12:01] danilos: I remember that this kind of change used to be a DB change. Did that change? ;) [12:01] http://paste.ubuntu.com/656349/ [12:01] ETOOMANYCHANGE [12:02] henninge, adding a new value to the enumeration isn't :) [12:02] Well, it has some similar consequences. [12:02] You have to be sure that various components of the system will cope with not knowing about it. [12:02] Which will normally make stuff OOPS. [12:03] wgrant: right, that has been accounted for [12:03] danilos: ok, if you say so. Don't blame me if it can't be merged .... ;-) [12:04] It's mergable, but it might break production :) [12:04] henninge, heh, fair enough [12:04] well, that's not that bad, then. [12:04] :) [12:04] :) [12:05] nah, this is a translation packaging job type, so if it breaks anything, it's going to break that, and that needs fixing anyway [12:05] I once tried that and PQM wouldn't merge it into devel. (I think that's where it failed) [12:05] but that is > 18 months ago. [12:05] henninge, wow, weird === benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: henninge, benji | Critical bugs: 240 - 0:[#######=]:256 [12:47] jml: Are you able to QA your PPA API thingy? [12:48] wgrant: I'm in the middle of something right now. you lining up for a deploy? [12:49] jml: Checking up on EU/US QA so I don't have to do much tomorrow, since I'm not allowed to. [12:50] wgrant: essentially, I can do it, provided someone else can grant & revoke commercial admin privs for me, but I would rather not. [12:50] k, I might just do it tomorrow, if nobody does it in the meantime. [12:50] Since we are 30 revs behind now :/ [12:51] yikes. === jtv is now known as jtv-zzz [12:59] wgrant: is this the create-as-private thing? [13:00] bigjools: Ja. [13:12] wgrant, jml: qa-bad I think. My private=False parameter is ignored. [13:14] I actually don't see any logic that sets private = True... [13:17] The whole point is to call it via the API [13:17] I mean, there is nothing in createPPA that uses the argument. [13:17] ... [13:17] Except for validatePPA. [13:18] yup [13:19] bigjools: Can you roll it back, please? [13:19] I'm sure a maintenance team can do it, I'm kinda busy [13:19] anyway it can roll out, it's not a blocker [13:20] Mmm. It's fairly misleading to have a 'private' argument that does nothing. [13:20] affects like 7 people [13:20] But I guess the set of affected users is small. [13:20] Yes. [13:20] oddly createPPA is only exported on version=beta [13:21]