[02:01] I'm starting to worry private projects has too much baggage to be any fun working on. [02:02] sinzui, are you around? [02:03] * StevenK stabs staging, the update script says that 11801 is the newest, which is a lie. [02:05] StevenK: staging doesn't have the new Wgrant-Optimized Update Path™ [02:05] :-( [02:06] wgrant: I'm just impatient, we have seven hours for staging to update. [02:11] * StevenK stabs the branch scanner too [02:16] deryck: I think it shouldn't be too bad once you actually cut through requirements, and everyone works out what needs doing. Disclosure has a similar aura around it, but it proves to be reasonably enjoyable while the path remains clear. [02:17] StevenK: Let's see what it's doing.. [02:17] wgrant, I hope you're right. I'm worried there's too much overlap between our two squads. [02:17] We may want to kill the current garbo/karma stuff to speed things along. [02:17] Depending on where it is. [02:18] StevenK: It started updating 4 minutes ago [02:18] StevenK: Should be qa-tagged in maybe an hour. [02:20] deryck: There is a lot. But we're almost out of that area. [02:20] Hopefully :) [02:21] wgrant, ah, that's encouraging then. [02:22] * StevenK stabs the branch scanner again [02:23] StevenK: What's it doing? [02:23] Not scanning ~stevenk/launchpad/bugsubscriptionfilter-itype [02:26] StevenK: It's cursed. You probably can't delete it. [02:26] StevenK: Rename and repush [02:26] StevenK: Did you link a bug manually just after you pushed it? [02:27] Yeah [02:28] Bleh [02:32] So you can't link a bug or create an MP or .... [02:32] Not on a Launchpad branch before the initial scan, no [02:33] * StevenK blames BranchRevision [02:33] Yes. [02:33] StevenK: This scan's not looking good either... [02:34] There we go [02:34] Success [02:34] The UI hasn't updated yet [02:34] StevenK: I actually meant you should rename the branch on LP, so you could keep a sensible name. [02:34] Has so [02:34] You're probably just enslaved. [02:37] FInally [02:37] s/I/i/ [02:37] wgrant: And I renamed them around [02:37] So I saw [02:38] wgrant: Do you want the review, or shall I pick on wallyworld_? [02:38] I shall. [02:38] While I wait for qastaging to QA for me. [02:38] Ah, you have qastaging do your QA for you? That sounds handy, you must tell me how. [02:41] StevenK: Did you consider refactoring _set_{statuses,importances,tags,information_types}? There's a lot of ugly common code there. [02:47] StevenK: 2012-08-02 02:45:49 INFO 2209-26-3 applied just now in 0.2 seconds [02:54] wgrant: I was just pondering how to do that. [02:54] wgrant: And thanks [03:03] wgrant: I'd love to refactor the three _set_ methods. I just can't think of a way to do it. [03:03] Hmmm, maybe [03:04] * StevenK scribbles some notes to fiddle with after lunch [03:04] StevenK: They all do a very similar thing. They might want to take a class, an attribute name, the current set of values, and the desired set of values. [03:04] wgrant: Yeah, but property() can't do anything with those [03:05] So _set_statuses will live on as a one line or so wrapper [03:05] Sure, that'd be a helper method. [03:05] Exactly. [03:05] Since I've got test passing, I'll try my hand at refactoring after lunching [03:27] lifeless: ping, around? [03:38] hi [03:39] lifeless: hey, wanted to beg/bribe for a LoC exception for the yui work that's blown up on me a few times. [03:39] first, fixing the buildout recipe upstream seemed dead since there's no upstream I can find and it's not maintained since 2010: http://pypi.python.org/pypi/plone.recipe.command/ [03:40] lifeless: so in stand up it came up about just doing the unzip work in a python script vs using the unzip dep in the buildout command [03:40] obviously, adding a python script to do the work is aLoC increase, but the goal of the work is to allow the YUI version feature flag to work and allow easily testing YUI/maintaining the JS state in the build directory [03:41] so wanted to see if I could convince you better for maintance and hopefully doesn't hang production servers on deploy this time :/ [03:41] not done yet, but start of the idea: https://code.launchpad.net/~rharding/launchpad/yuiv4/+merge/117800 doing in spare time around the private project requirements stuff [03:43] whats the copytree for ? [03:44] lifeless: the .zip has test/doc dirs we don't want. We only care for the build dir. so we extract to a build/js/tmp and copytree the build dir to build/js/yui-xxx [03:45] can't you just pass in the dir you want ? [03:45] hmmm, doesn't let you transform it [03:45] sadface. [03:45] zipfile will let me extract based on name, but I need to have all the names I want not a single dir adjusted [03:45] ok [03:46] rick_h_: so, plone.recipe.command is still buggy here, I'm worried about us using it. It is a booby trap. [03:47] lifeless: true, I wanted to ditch it but don't see how to get the version into the here from versions.cfg and such directly [03:47] rick_h_: I think the loc overhead is tolerable, but that about 80% of the overhead is because you're calling a process not a function. [03:47] 'version into the here' ? [03:47] lifeless: so yui-default gets the version from versions.cfg, passed into the call to this script via the buildout section [03:48] my first thought was to use the buildout generation to hard code the yui version into the script when the template is extracted [03:48] but that only gives me one version, not multiple for the feature flag setting [03:49] versions.cfg talks about python packages [03:49] yui isn't a python package. [03:49] How is it relevant ? [03:49] yes, but part of the reason my first idea was turned down was that it didn't respect versions.cfg. There's an entry in versions.cfg for YUI that is used to calc the 'default' yui everyone gets [03:50] why? [03:50] because it's a nice single place to specify the correct version of a dependency? I'm only guessing at the original reasong [03:50] reasoning that is [03:51] if it wasn't for that I'd turn the whole thing into a make target and skip buildout entirely, but that versions.cfg traps me into the use [03:51] don't use versions.cfg [03:51] easy done. [03:51] ok, nvm then. I should be able to continue to use unzip from the makefile then, ditch the plone.recipe.command all together, and rework this [03:51] seriously, versions.cfg is for python package versions, its not rich enough to describe '4 versions of a package' [03:52] because python doesn't support concurrent loading of one API [03:52] right, that's why the buildout is set to run yui-default and yui-3.5 [03:52] so to wedge this into versions.cfg, you'd have to use N different packages [03:52] lifeless: ok thanks, works for me. [03:52] and do string matching to decide which ones were YUI [03:52] -> thats more than a little horrid [03:53] well, it appears to populate a var yui_version = ${versions:yui} [03:53] so it didn't seem quite that dirty, but understand [03:55] I see 'yui = 3.3.0' in versions.cfg [03:55] what am I missing ? [03:55] lifeless: right, the yui-default block in buildout.cfg is getting that via ${versions:yui} [03:56] sure, but this does demonstrates the friction. [03:56] I don't think you're missing anything, just that we're only using it to describe the 'default' version of the package and using additional buildout.cfg blocks to load the feature flag available versions of YUI for side by side install [03:56] You're encoding versions in buildout.cfg [03:56] not in versions.cfg [03:56] lifeless: correct, lots of friction [03:58] lifeless: right, ok. I'll rework it in my spare time and make an effort to ditch the plone.recipe.command [03:58] thanks for the time/chat. [03:58] anytime [03:58] long term the mythical handling-js system will want this flexability [03:59] yes, with 3.6.0 released today it's killing me I've not gotten it sorted yet. [04:07] wgrant: 2 files changed, 40 insertions(+), 75 deletions(-) [04:10] StevenK: Excellent. [04:10] wgrant: http://pastebin.ubuntu.com/1124649/ [04:12] StevenK: _get_collection, _set_collection, mabe? [04:12] maybe [04:16] wgrant: _set_collection makes the line 78 lines [04:16] Er, chars [04:16] That's fine. [04:16] I thought 77 was the limit? [04:17] 80 [04:17] now [04:17] Or was it 79 [04:18] One of those two [04:18] Certainly not 77 [04:18] /usr/share/pyshared/pocketlint/formatcheck.py:DEFAULT_MAX_LENGTH = 80 [04:18] sinzui has spoken. [04:20] wgrant: Right. Do you want to see the new diff? [04:21] StevenK: That would be lovely. [04:21] wgrant: http://pastebin.ubuntu.com/1124660/ [04:23] Now it's a little easier to see the differences :) [04:27] wgrant: Shall I push that up? === Ursinha` is now known as Ursinha [04:29] StevenK: Might as well. [04:29] Less good is better code. [04:29] Er [04:29] Less code is better code. [04:29] Yes, I'm not sure I want to refactor _get_tags() and friends [04:30] I'm tempted to kill the pointless docstrings [04:31] wgrant: Opinion? [04:34] Maybe [04:36] wgrant: Killing the docstrings turns it into +39/-100 [05:02] * StevenK stabs Firefox [05:05] lifeless: So I'd like to organise an FDT tonight with two patches, 2209-26-{2,3}. [05:06] lifeless: 26-2 updates two tables, BugSubscriptionFilter{Status,Importance} and 26-3 creates a new table. [05:06] More specifically, 26-2 changes from a serial to a natural primary key [05:08] lifeless: Timing on staging puts both patches applying in 0.5 seconds. [05:14] * StevenK sorts out the db-stable deployment report [05:14] wgrant: Do we want a NDT? [05:15] I'd like to make sure I've not broken get_bug_tags() with r15733, but I'm not sure which pages make use of it. [05:24] StevenK: Let's see. [05:26] StevenK: Huh [05:26] wgrant: Hm? [05:26] StevenK: I may have missed something. What do you see as the callsites for it? [05:26] I can only see one, but I'm presumably missing something. [05:28] wgrant: IDistroseries.getUsedBugTags(), IDistribution.getUsedBugTags(), IProduct.getUsedBugTags() at least [05:28] IProductSeries and IProjectGroup too I think [05:28] Right. And what uses those? [05:29] Yeah [05:29] That whole set of methods can just be deleted [05:29] Only one callsite [05:30] (BugEditView uses it to decide whether a tag is new. But the JS doesn't do it, so the classical form doesn't need to either) [05:30] So I fixed them for nothing? :-) [05:30] Well [05:30] It led us to them [05:30] So now we can destroy them. [05:31] Which is the callsite I need to preserve? [05:31] There is only one callsite that I can see, BugEditView.validate [05:31] And you don't need to preserve it [05:33] Uh? [05:33] lib/lp/registry/model/distributionsourcepackage.py: return self.distribution.getUsedBugTags() [05:33] lib/lp/registry/model/sourcepackage.py: return self.distroseries.getUsedBugTags() [05:33] I'm pretty sure those are just delegations [05:33] But I haven't checked. [05:33] lib/lp/bugs/browser/bug.py: bugtarget.getUsedBugTags() + bugtarget.official_bug_tags) [05:33] def getUsedBugTags(self): [05:33] """See `IBugTarget`.""" [05:33] return self.distribution.getUsedBugTags() [05:34] StevenK: Right, that one is the only real callsite [05:34] And it can die [05:35] wgrant: It can? [05:35] Look at what it does. [05:35] (ignoring the XSS hole) [05:36] So the entire validate method can just die horribly? [05:36] Right [05:36] You might have noticed the new tag validation on the AJAX widget. [05:36] It's very fancy. [05:37] So fancy that it's invisible and 0 bytes in size. [05:37] Hahaha [05:37] Since everyone has used the AJAX widget since 2009 and not lamented this warning's absence, I think it can be abolished. [05:37] as a bonus, you get rid of shitty code [05:37] And an XSS vulnerability or two [05:39] There's also a not insignificant volume of tests. [05:39] So whoever does this gets a nice LOC credit. [05:40] StevenK: got a bazaar.l.n link to those two patches ? [05:41] lifeless: Sure, give me a tick. [05:41] StevenK: :( you stole my patch series [05:41] http://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-devel/view/head:/database/schema/patch-2209-26-2.sql http://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-devel/view/head:/database/schema/patch-2209-26-3.sql [05:42] 2209-26-* was a new series so the BVP stuff was grouped :( [05:42] I didn't steal them, they were freee [05:46] lifeless: ^^ the two patches I linked above [05:47] wgrant: 13 files changed, 5 insertions(+), 150 deletions(-) [05:47] ok [05:47] StevenK: You've grepped around for tests? [05:47] StevenK: I'm curious why a new table not an array [05:47] And any other orphaned methods? [05:48] lifeless: I suggested a new table, to maintain consistency and share code. [05:48] lifeless: But indeed, all four filters deserve to be arrays [05:48] And should probably be migrated that way eventually. [05:48] wgrant: Yeah [05:49] steven@undermined:~/launchpad/lp-branches/destroy-getusedbugtags% bzr grep getUsedBugTags | grep -v getUsedBugTagsWithOpenCounts | wc -l [05:49] 0 [05:49] StevenK: you have+1 on double-dipping the FDT. [05:49] lifeless: Thanks [05:50] wgrant: So I guess FDT for r11803 [05:50] StevenK: You also removed confirm_tag_action? [05:50] Nope [05:51] And indeed, 11803 is correct [05:51] But you should remove confirm_tag_action [05:51] _confirm_new_tags = False [05:51] can die as well [05:51] And render() [05:51] And __init__ [05:51] And part of the template. [05:52] s/part/all/ [05:52] That view appears to be nearing its end... [05:53] Which template? [05:54] The view's template. [05:54] It's a form [05:54] Except for the notifications [05:54] Except we've removed the notifications [05:54] So it's a form. [05:55] Oh [05:55] hm [05:55] So I drop the template from the ZCML? [05:55] The action can go away as well [05:55] Since that just makes it a LaunchpadEditFormView [05:55] So the view ends up with like one method [05:55] Right [05:55] Um [05:55] Maybe [05:56] May need to use generic-edit-form.pt or whatever it is [05:56] template="../../app/templates/generic-edit.pt"/> [05:58] wgrant: 15 files changed, 9 insertions(+), 230 deletions(-) [06:00] StevenK: Make sure to set next_url properly [06:00] I'd rename cancel_url to next_url, then set cancel_url = next_url [06:00] Already done [06:01] So the view's like 10 lines and uses the generic template now, right? [06:09] wgrant: Hmmm, it is 12 [06:10] BugMarkAsDuplicateView had a little surgery too [06:12] StevenK: What'd you do to it? [06:13] wgrant: It had next_url specially due to BugEditView, now that it doesn't require the specialness, it can be pushed up to the base class [06:14] StevenK: Ah, right [06:14] StevenK: However, I've just noticed what may be a fatal flaw in this simplification [06:14] StevenK: They're views on BugTask [06:14] But they operate on the Bug [06:14] That's why they're not classical LEFVs, I suppose. [06:14] So you may need to preserve the custom actions. [06:16] Hmmmm, not sure. [07:06] wgrant: 5 failed tests [07:06] StevenK: :( [07:08] A few of them look caused by my overzealous deletion [07:08] Nah [07:08] No such thing as overzealous deletion [07:08] Haha [07:08] It just means you haven't deleted enough tests [07:08] => underzealous deletion [07:31] good morning [09:00] hi, could someone help me? I'm trying to build the language packs for 12.04.1, from a script running on a server which calls LP (the getPackageUploads API call) to fetch some translations files. However this seems to fail with a timeout, I've tried a couple of times with the same end result. Here is the log from the script, and the LP response. Perhaps the 'x-lazr-oopsid: OOPS-80db179b422b1ae941c44605effcdaf1' could be helpful in determining what's hap [09:00] pening? [09:00] Any help will be greatly appreciated, as this is currently blocking the 12.04.1 language pack build [09:03] adeuring: wgrant StevenK anyone able to help dpm out [09:03] thanks czajkowski [09:04] sorry, forgot to paste the log, here it is: http://pastebin.ubuntu.com/1124842/ [09:04] This is probably fallout from cjwatson's work, but let's see... [09:06] I'm running the script again, to see if it runs through or if I get another timeout [09:30] wgrant, have you had a chance to look at the timeout issue I was mentioning (not sure if you meant you were looking at it, just checking)? ^ [09:31] dpm: Oh, I think we got netsplit [09:31] 19:10:23 < wgrant> Yeah [09:31] 19:10:35 < wgrant> The new PackageUpload security adapter is issuing several queries for every item. [09:31] 19:10:36 < wgrant> Um [09:31] 19:11:05 < wgrant> dpm: Where's this script? [09:33] wgrant, ah, yeah, hadn't seen the ping. So the script is at http://bazaar.launchpad.net/~ubuntu-langpack/langpack-o-matic/main/view/head:/import and the relevant bit that calls the function is at http://bazaar.launchpad.net/~ubuntu-langpack/langpack-o-matic/main/view/head:/lib/static_translations.py#L58 [09:35] webops: Who is my willing vict^W^Wfriendly webop for the FDT in 25 minutes? [09:35] Sigh [09:36] StevenK: I am [09:47] dpm: Ugh, that's pretty awful. Grabbing like 120000 uploads from LP every time. [09:47] Let's see. [09:47] ouch [09:51] Ah, it's actually several times more than that. [09:57] wgrant, anything that can be done either in LP or in the script to improve on that and avoid the timeout? [10:19] dpm: Sorry, will get back to you soon, just had a bit of a crisis. [10:19] wgrant, no worries, thanks for your help [11:56] jam: re. dpm's issue, I'd like to bump the timeout for DistroSeries:EntryResource:getPackageUploads from 5s back up to 9s, as it was a couple of months back. [11:57] It'll hopefully be enough. [11:57] Just. [12:02] wgrant, I assume that even if the timeout is reverted in LP, it will still take a while for it to land. Is there anything I can do on my side (i.e. on the script), so that I can start building the language packs in the meantime? [12:04] dpm: We can apply the timeout change instaneously. [12:04] Once I find a TL to approve it. [12:05] wgrant, ah, awesome, that'd work best for me then if it's possible [12:16] Oh look, there's one. [12:52] wgrant: it seems a pretty major band-aid, since once he needs another 10% more data it will hit the timeout again, but if it unblocks him for now, then it seems ok in the short term. [12:52] Is there a way to comment the change so we know who needs the time bump? [12:53] jam: Well, it's batched [12:53] jam: Batches of 40 work atm [12:53] Batches of 75 do not [12:54] An O(batch size) query count is the problem. [12:54] And it's easier to set a feature flag on LP temporarily than hack launchpadlib or the script [12:54] wgrant: it is batched inside launchpadlib? [12:54] jam: Right, iterating over a collection retrieves it in chunks [12:59] wgrant: so do we have a decent way to document the decision, so we can revisit it later? [13:00] or is there at least a bug about DistroSeries:EntryResource:getPackageUploads being O(batch_size) ? [13:00] if so, it seems fine to bump the timeout [13:01] jam: I'm going to talk to cjwatson about it, but there indeed needs to be a bug. [13:01] jam: We can mention in the feature flag changelog that it's because of this script [13:02] * wgrant is filing [13:07] Bug #1032176 [13:07] <_mup_> Bug #1032176: DistroSeries.getPackageUploads timing out due to authorization checking

< https://launchpad.net/bugs/1032176 > [13:07] jam: So, I propose to add the feature rule 'hard_timeout pageid:DistroSeries:EntryResource:getPackageUploads 120 9000' [13:08] wgrant: what is the 120? [13:08] jam: The priority [13:08] We currently have up to 119 [13:08] https://launchpad.net/+feature-rules [13:08] wgrant: seems fine to me [13:09] jam: Thanks. [13:15] dpm: Care to try the script again? [13:16] The timeout is up and the method call works for me now [13:19] jcsackett: we are going to hangout with Orange and 16:00. I want their help to solve sharing parts of projects [13:43] wgrant, sure, let me start the run, thanks! [13:43] running now [13:44] last time it stopped after ~10 minutes so I guess if it runs longer than that that'll be some indication that it works [13:44] sinzui: i assume from the use of 24 hour time you mean UTC? [13:44] jcsackett: yes, sorry for the ambiguity [13:44] sinzui: no worries. :-) [14:52] hi folks. we have a rather urgent problem with a spammer: https://answers.launchpad.net/launchpad/+question/204862 [14:52] can anyone help us? [14:54] barry: sure [14:55] barry: done [14:55] czajkowski: thanks! i know we'll continue to play whack-a-mole with this person. this isn't the first time and i'm sure it won't be the last. [14:56] i don't know why he hates us ;) [14:56] meh always one [14:56] just file a new answer each time and we can track it [14:56] czajkowski: thank you [14:57] np === abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2 [15:30] adeuring: There's no OCR. Could you please review https://code.launchpad.net/~abentley/launchpad/better-find-missing-ready-error/+merge/117926 ? [15:30] abentley: sure [15:41] abentley: r=me, minor nitpick [15:41] adeuring: Thanks. === jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: jcsackett | Firefighting: - | Critical bugs: 4.0*10^2 [15:58] abentley: apologies for not having set the topic earlier. [15:59] jcsackett: I'm a slacker too. I was OCR yesterday and today I was still listed in the topic. [15:59] the topic is the easiest part to forget. esp if you get in the habit of just checking the activereviews page all the time. === almaisan-away is now known as al-maisan === al-maisan is now known as almaisan-away === deryck is now known as deryck[lunch] [18:30] o/ === deryck[lunch] is now known as deryck [19:57] deryck: ping [19:57] rick_h_, hi [19:57] deryck: hi, so I can do a call whenever [19:57] deryck: got a sec to chat before I EOD? [19:58] rick_h_, on call, sorry. [19:58] deryck: np [21:20] lifeless, sorry, crazy afternoon. Let's chat next week. Too much churn right now with what's going on with purple too. [21:21] ok [21:44] irssi [21:45] indeed [22:02] * jcsackett just now realizes that the "dead" irssi window he was restarting wasn't dead. === jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2