[00:00] wgrant: blacklist +1 [00:00] lifeless: But that means having the list of exclusions in a separate tree. [00:00] wgrant: we've needed one exclusion in 6 years [00:01] wgrant: I think we can deal [00:01] True. [00:01] Pushing. [00:03] Ah, allenap beat me to a bulk loading helper. [00:03] * lifeless isn't sure its the right way [00:04] I'm not sure it is either. [00:04] we want, for instance, to eager load visibility rules etc [00:05] so I think it needs per-type handling rather than a separate helper [00:05] however, allenap has written something that will be usable for somethings [00:05] Right, that's what I was thinking. It is a pattern that we already use in several places. [00:06] And this helper can do nice stuff like not issuing a query if everything is already preloaded. [00:06] it introspects the storm cache? [00:06] Since that is going to become a common case, I think. [00:06] *can do* [00:06] Not does now. [00:06] ah [00:06] sure [00:07] arse [00:07] leonardr: I don't suppose you are still around? [00:07] leonardr: I have a lazr-restful problem [00:07] thumper: i'm here [00:07] ah, awesome [00:07] leonardr: let me pastebin something... [00:08] lifeless: https://code.launchpad.net/~wgrant/launchpad/exclude-lpnet-template/+merge/52497 [00:10] Thanks. [00:10] lifeless: You're landing lamont's thing and the staging fix? [00:11] thats how I found this, yes. [00:11] I'm not sure what branch the check checks out [00:11] Checking. [00:11] leonardr: http://pastebin.ubuntu.com/577252/ [00:11] ./launchpad bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable [00:11] fuuuu [00:11] leonardr: getting TypeError: Only a read-only field can have a mutator method. [00:12] lifeless: But you can fix that in the branch :P [00:12] wgrant: meh, will wait for bb [00:12] leonardr: wanting the accessor and mutator only in "devel" [00:12] stable is not unreasonable [00:13] leonardr: any ideas? [00:16] reading [00:17] thumper: have you tried putting @operation_for_version first? [00:17] leonardr: first as in at the top, or bottom? [00:17] at the bottom [00:17] * thumper tries [00:18] nope, that doesn't fix it [00:18] right now the mutator is exported in beta nad the field is not exported at all in beta [00:18] same error? [00:18] yep [00:19] thumper: the field itself needs to be read-only. it can't just be exported as read-only [00:19] oh? [00:19] see the mutator_for method in lazr.restful declarations.py [00:19] really? [00:19] that's what it says [00:19] if not self.field.readonly: [00:19] raise TypeError("Only a read-only field can have a mutator " [00:19] "method.") [00:19] hmm... [00:19] we should fix that [00:19] maybe... [00:20] Why can't you make the field read-only? It only affects forms and the webservice. [00:20] sinzui: are you going to land the new lazr restful egg? [00:20] wgrant: because the forms edit it [00:21] wgrant: I'd have to override the field... [00:21] :( [00:21] wgrant: and it changes the permissions [00:21] wgrant: if the field is readonly, the permissions don't allow setting [00:21] It doesn't affect security proxies. [00:21] lifeless: i think it would be okay to let mutator_for succeed if the field is never *published* as read-write in any version for which it has a mutator [00:22] Hmm, unless maybe it affects set_schema, but not much uses that. [00:22] * leonardr has to go [00:22] wgrant: I think it does... [00:22] leonardr: ok [00:22] thumper: Anyway, shouldn't the form be using the mutator anyway? [00:23] wgrant: it means more hoop jumping to extract the field from the data dictionary, getting the delta object first, raising the object modified event manually [00:23] man I wish our form infrastructure used the API :) [00:25] * thumper has a plan [00:26] :( [00:27] ohh... tentative success [00:35] :( now getting You tried to modify a read-only attribute. in the 400 response [00:41] checkwatches OOPSes and bye-bye-nullbugtask today, I think. [00:43] wgrant: but its perf tuesday! [00:43] lifeless: Killing NullBugTask gets rid of some timeouts! [00:43] But OK. [00:44] Maybe getBuildRecords. [00:44] \o/ [00:44] I also need to attack +copy-packages again.. the copying is now fast, but the rest of the page is not :/ [00:45] wgrant: its a never ending story [01:00] thumper: it sounds like the lookup of the mutator operation didn't succeed. you could try putting a breakpoint in generate_entry_adapters() (lazr.restful declarations.py) [01:00] it should be creating a PropertyWithMutator for the field in the appropriate version [01:01] i'll try that [01:02] * thumper sighs [01:02] every freaking entry [01:04] ok [01:04] lots of continues, and found the right interface [01:16] huwshimi: Bug 730535 [01:17] <_mup_> Bug #730535: Enter key ignored the first time in bug tag editing field < https://launchpad.net/bugs/730535 > [01:17] huwshimi: is that a regression from your changes? [01:18] lifeless: it looks like the same issue. I believe my changes won't be rolled out until lazr-js is updated in LP. [01:19] huwshimi: ah! then your bug should not be closed [01:20] huwshimi: and this new bug should be a dupe. [01:20] lifeless: yeah [01:20] lifeless: I think it was marked as closed during a deployment or something [01:22] * thumper sighs [01:22] huwshimi: do you know how to move forward on this? [01:22] lifeless: Nope :) [01:22] ok [01:23] first things first, lets get the bugs sorted out [01:23] find your old one [01:23] set to in progress [01:23] dupe the new onto the old [01:23] the old one should have two tasks, one on lazr-js and one on launchpad,t he lazr-js one should be fix released if your change is merged into it. [01:24] I'm moderately sure deryck finished upgrading us to the latest lazr-js recently so your fix should be easy to get to. [01:25] Ah, sorry, that was probably my fault. I thought I set it back, though :/ [01:32] lifeless: The old one (bug #580404) doe not have a lazr-js task, I assume I can just add that? [01:32] <_mup_> Bug #580404: pressing enter in tags field doesn't save them

< https://launchpad.net/bugs/580404 > [01:33] huwshimi: yes [01:33] huwshimi: you made the change in lazr-js right ? [01:37] lifeless: ugh, sorry I'm getting random lockups with my keyboard and mouse (but I can see that the OS is still working) [01:37] lifeless: Yes the changes were made to lazr-js [01:38] Yippie, build fixed! [01:38] Project db-devel build #427: FIXED in 5 hr 14 min: https://hudson.wedontsleep.org/job/db-devel/427/ [01:42] lifeless: ok, I've done all that. What next? [01:42] next, you need to figure out whats new in lazr-js leading up to the commit where your patch landed, so you can decide how much qa you want to do [01:43] and/or whether you want to land everything or just up to your patch [01:43] use bzr and or bazaar.launchpad.net to look at the history and decide [01:43] pick the rev of lazr-js you want to update to [01:43] and then we go to the actually-doing-it step [01:44] lifeless: ok thanks will take a look [01:46] lifeless: How do I know what hasn't been landed? [01:47] lifeless: Or rather released [01:47] look at versions.cfg, see the version there, look at the branch for lazr-js, walk back until the release [01:51] lifeless: I'm not entirely sure what you mean by that, there are a lot of version numbers in that file and none seem to refer to lazr-js? I may be completely mis-understanding you [01:52] lazr-js = 1.6DEV-r202 [01:53] OOPS reports during release week are depressing. [01:53] wgrant: because deploys stop ? [01:53] https://lpstats.canonical.com/graphs/AppServerRequestLpnet [01:54] 14:51 < huwshimi> lifeless: I'm not entirely sure what you mean by that, there are a lot of version numbers in that file and none seem to refer to lazr-js? I may be completely mis-understanding you [01:54] 14:51 < lifeless> lazr-js = 1.6DEV-r202 [01:54] lifeless: Sorry crashed again [01:54] lifeless: Yes. [01:55] lifeless: Ah right, you're referring to the versings.cfg in Launchpad, not the one it lazr-js :) [01:56] lifeless: ok, so there is one other commit since the last release [01:57] * 1954 Exceptions [02:00] lifeless: The other commit does not have an attached bug, is it still possible for me to be able to qa it somehow? [02:01] huwshimi: of course [02:01] bugs don't permit qa, they are just about recording stuff [02:02] huwshimi: make an assessment of the other change [02:05] lifeless: It's a module that's been updated (looks like a file has been replaced with a whole new version). [02:17] ok next time this crashes I'm giving up and going back to my old laptop [02:17] huwshimi: what is it? [02:17] lifeless: the laptop? [02:17] huwshimi: so you need to decide if its safe to include in lp as-is [02:17] yes the laptop [02:17] it's a 2011 Macbook Pro. [02:18] lifeless, and now? [02:18] I forgot to change a link [02:18] sorry [02:18] Ursinha: brilliant [02:20] ok that time it was Natty crashing. Laptop, I'm giving you one more chance [02:29] lifeless: from a quick grep I can't find any references to that lazr-js module. [02:29] where ? ;) [02:30] lifeless: in the launchpad codebase [02:30] lifeless: I could be wrong though :) [02:32] huwshimi: so if we're not using galery-form, its a no brainer [02:32] huwshimi: I think its on the wiki somewhere, but what you need to do next is make a new sdist, put it in the lp download cache (copy it in, bzr add, bzr commit), update versions.cfg in lp and propose that as a branch to land in devel [02:33] lifeless: Where do we define which javascript files we include? [02:37] * thumper leaps through a few more hoops [02:38] huwshimi: I don't know [02:59] lifeless: So I'm wrong about us not including that file [03:02] huwshimi: in which case, check that lp will still work with the new version :) [03:04] lifeless: And how do I do that? :) [03:05] I'm sure you'll think of something [03:05] e.g. how did you make sure the bug tag editing worked [03:06] lifeless: Well I'm trying to figure out if we actually use this module at all [03:17] wgrant: do you remember where we set allowed attributes for built in types? [03:18] thumper: ^ [03:19] for built in types? [03:19] like what [03:19] ? [03:19] dict [03:19] for instance [03:19] lifeless: hmm... [03:19] in this case, defaultdict [03:19] no [03:19] doesn't have __getitem__ permitted [03:20] grep is your friend :) [03:20] and making an interface would be batshit [03:20] it'll be in a configure.zcml [03:22] Does anyone here know how we handle which javascript files are included in Launchpad? I've found this, but it doesn't seem to be correct: https://dev.launchpad.net/JavaScriptBuildSystem [03:23] huwshimi: what do you want to know? [03:23] I've learned some things [03:24] lp_sitecustomize.py [03:24] thumper: There is a javascript file (as part of lazr-js) that is included in our launchpad.js, but I can't see where we actually include that to be built. [03:25] jsbuild is full of magic [03:25] and a PITA [03:25] make jsbuild calls out to a couple of utility scripts to determine dependancies [03:27] thumper: do you remember which scripts they are? [03:28] huwshimi: look in the Makefile [03:28] for the jsbuild: target [03:28] they are in backquotes [03:28] The file list is hardcoded now, isn't it? [03:30] In utilities/yui-deps.py [03:32] wgrant: does that include the lazr-js bits? [03:34] huwshimi: are you doing something in lazr-js? [03:34] huwshimi: I have a request [03:35] thumper: Ah, no, just all of YUI :( [03:36] thumper: I'm trying to build lazr-js for Launchpad and trying to determine if we are using a module that was updated to a new version by someone else. [03:36] thumper: What's the request [03:36] ? [03:36] huwshimi: somewhere in the picker code, it is adding a [03:36] huwshimi: it needs to die, and CSS used instead [03:39] thumper: I don't think I want to change anything else right now or else my brain will melt, but feel free to file a bug or something and I'll try and get to it :P [03:39] :) [03:39] ok [03:45] thumper: if you landed a branch quickly I could probably include this in the build. [03:45] huwshimi: ETOOMUCHTODO [03:45] thumper: Fair enough :) [03:46] huwshimi: I've already been on a diversion [03:49] anyone: https://code.launchpad.net/~thumper/launchpad/choice-widget/+merge/52357 [03:52] thumper: Those scripts don't seem to mention lazr-js stuff [03:53] thumper: or am I missing something? [03:54] lib/canonical/launchpad/icing/lazr/build/lazr.js [03:54] huwshimi: that is the last line of the jsbuild command in the Makefile [03:54] huwshimi: I'm guessing that you'll find that is a symlink to the location where lazr-js is built [03:55] What's the setuptools sdist option to add a version suffix? [03:55] using the jsbuild_lazr make target [03:55] wgrant: I don't remember but I need it too [03:55] wgrant: what are you packaging? [03:55] thumper: Updating pydkim. [03:55] * wgrant greps logs. [03:56] not in my history sorry [03:56] Because it's not in --help :( [03:56] -b [03:57] thumper: Ah right, so we do just include the whole of lazr-js and don't control which files we include. [03:57] huwshimi: maybe... [03:58] thumper: Thanks for that. Now I can get on with my investigating. [03:58] Except that the egg_info command doesn't work here :/ [04:00] It's like setuptools isn't installed... [04:22] lifeless: so I think after all that we include that file but don't use it, but I can't be sure [04:23] Does anyone want to fix the failed stable into db-devel merge? [04:23] Not again :( [04:23] Fixing. [04:23] wgrant: Thanks [04:23] This is changing one that I already resolved on Saturday :( === MTecknology is now known as MT-TakinBreak [04:36] wgrant: Can you have another look at https://code.launchpad.net/~stevenk/launchpad/populate-spr-changelogs/+merge/52363 ? I've addressed all of your concerns, bar IMasterStore, since everything else done by garbo uses it. [04:37] StevenK: You renamed getCandidateSPRIDs, but it still returns SPR IDs... [04:38] Also, why are we still restricting to Debian? [04:39] 1. Our sampledata sucks. 2. I don't think there is any point doing it over more. [04:39] Apart from that it looks good. [04:39] What about all the Ubuntu publications from more than a year ago? [04:39] wgrant: Er? It returns a ResultSet of SPRs ordered by id? [04:40] No, it returns a ResultSet of SPR.id ordered by id. [04:40] Bah, fixed [04:41] wgrant: I'm happy to say upload_archiveID IN (ubuntu.main_archive.id, debian.main_archive.id) [04:41] StevenK: Why say that at all? [04:41] Running it over sampledata which the tests is horrid [04:41] Er. Which the tests do [04:42] It's easy to clean the sampledata up. [04:42] I'd rather just delete it :-) [04:42] Right, but deleting it in the tests is hard. [04:42] lifeless: What should I do with this then? I don't want to release these changes if they'll break things... [04:43] huwshimi: will they break things? [04:44] lifeless: Well that's the problem, I don't know. I can't see where we're using this module, but that doesn't mean we're not using it. I could just be looking for the wrong things. [04:45] wgrant: Personally, I'd prefer to leave it for Debian. For the derived case, we aren't going to care about the older Ubuntus [04:45] StevenK: There is stuff in Natty that has no changelogs... [04:45] huwshimi: check the wiki for info, failing that ask for help: mail the dev list asking for 'how to find out where a lazy-js module is used by lp' [04:45] Really? [04:45] lifeless: OK thanks, will do. [04:46] StevenK: Anything that was uploaded before we started importing changelogs not-very-long-ago. [04:46] wgrant: The DB on DF for maverick said zero had a null changelog ... [04:47] StevenK: With upload_distroseries=natty? Sure. [04:47] But not much stuff in natty was uploaded to natty. [04:47] The natty DB on DF is very little [04:47] Even moreso there. [04:47] StevenK: the point is the things uploaded to maverick that haven't been changed in natty [04:48] wgrant: So, how do you propose I have the tests ignore the sampledata? [04:48] StevenK: Expire all LFAs at the start of the test. [04:48] The brutal approach. [04:48] Allow me to say "Ew" [04:48] It is better than sampledata. [04:49] * StevenK tries to work out how to do so [04:49] We may eventually want two template DBs, one without crap. [04:50] Since we won't be able to drop the crapful one for a long time :( [04:54] wgrant: I can't think of an elegant way to expire all the LFAs [04:55] StevenK: Store.find(LibraryFileAlias).set(content=None)? [04:59] wgrant: Having it return SPRs directly complains about the GROUP BY [04:59] StevenK: Group by SourcePackageRelease instead. [04:59] Hmm. [05:00] That might be why I was grabbing IDs... grouping by all the fields may have proved slower, although that seems odd. [05:00] Since it should be able to optimise to a pkey grouping. [05:00] shoulda coulda woulda [05:00] the way to do that is to use the id based query as an inner query [05:01] like e.g. _getExternalMessages does [05:17] StevenK: can you think of any reason why gina would initialize the references in SPPHs it creates using the ids for distroseries, SPR, etc. instead of the actual model objects? [05:18] Nope [05:18] gina is .... special [05:21] * StevenK grumbles [05:21] jtv: code fragment ? [05:21] Now it dies due to a variable used before assignment :-( [05:22] And if I change its indent level, it loops until killed [05:22] lifeless: [05:22] entry = SourcePackagePublishingHistory( [05:22] distroseries=self.distroseries.id, [05:22] sourcepackagerelease=sourcepackagerelease.id, [05:22] status=PackagePublishingStatus.PENDING, [05:22] component=component.id, [05:22] section=section.id, [05:22] datecreated=UTC_NOW, [05:22] datepublished=UTC_NOW, [05:22] pocket=self.pocket, [05:22] archive=archive [05:22] ) [05:22] StevenK: is gina special enough to warrant that? [05:23] I have a branch which fixes that. [05:23] jtv: I would change it to distroseriesID=self.distroseries.id etc [05:23] jtv: Er, things like that are the REASON why gina is special [05:23] we shouldn't overuse Reference objects, they are fragile [05:23] Okay, but is there any justification? [05:23] For manually instantiating it, rather than using PublishingSet.newSourcePublication? [05:24] No, for using the ids instead of the model objects. [05:24] I suspect SPPH support was bolted on. [05:24] Because I'm trying to make it use newSourcePublication. [05:24] https://code.launchpad.net/~wgrant/launchpad/unify-publication-creation [05:24] One remaining issue: it doesn't set datepublished. [05:25] Which is fine [05:25] We don't want it to [05:25] It shouldn't even do that in the first place. [05:25] Perhaps so, but inconsistent. [05:25] Shame that branch sat around for half a year—it's a change I need for my current work. [05:26] jtv: I didn't want to get into the gina argument at that point. [05:27] But all in all we don't really know whether there's any particular reason to use ids there? [05:27] There isn't one. [05:28] If you see something odd in gina, it's because gina is crap, not because it needs to be that way. [05:28] That's nice to hear. [05:29] Brings me to the next question: newSourcePackagingPublication creates a new DSP if one does not exist—which is breaking gina tests. [05:29] I added a permission in that branch. [05:30] I think it's the right thing to do. [05:30] I was wondering if maybe it was just laziness of the test, and there should always be a DSP [05:30] gina predates DSP. [05:30] There's no problem with creating one. [05:31] wgrant: Why didn't you want to get into a gina argument? [05:31] StevenK: Because nobody knows whether we want to stop setting datepublished or start creating them Published or burn gina to death. [05:32] whats gina used for? [05:32] Julian said yesterday we want to stop setting datepublished [05:32] OK, how does _'s string interpolation work? [05:32] lifeless: It imports Debian [05:32] lifeless: Importing Debianish archives into the DB and librarian. [05:32] And so the datepublished=UTC_NOW is completely bogus [05:33] jtv: It is a lie, and not one with considerable utility. [05:33] Kill it. [05:33] I had already done that. [05:33] wgrant: http://pastebin.ubuntu.com/577330/ makes me sad [05:34] StevenK: Oh? [05:34] It makes me happy. [05:34] http://pastebin.ubuntu.com/577331/ [05:35] StevenK: Which line does it fail on? [05:36] self.start_at = spr.id + 1 [05:36] Ah, that's not a new issue. [05:37] It'll happen whenever there are no SPRs to process. [05:38] :-( [05:50] 200 tasks -> 10 second renders :( [05:51] lifeless: But it at least renders now? [05:51] thats on my laptop [05:51] Ah. [05:51] which is an i7 [05:51] Ow. [05:53] 50ms per bugtask hmm === almaisan-away is now known as al-maisan [05:58] lifeless: Is wampee running with the second new set of appservers? [06:00] wgrant: yes [06:00] Excellent. [06:05] wgrant, want to review my branch? https://code.launchpad.net/~jtv/launchpad/bug-730460-use-spph-factory/+merge/52515 [06:05] jtv: Gladly. [06:05] Thanks. [06:10] wgrant: I split off the factory part to free me up for the real work, so there's not much there. [06:11] lifeless: Could you mentor that? === al-maisan is now known as almaisan-away [06:21] I'll go grab some food. === jtv is now known as jtv-eat [08:09] lifeless: What would you recommend as a Python testrunner these days? I normally use nose, but you probably have better ideas. === almaisan-away is now known as al-maisan [08:18] wgrant: testr with subunit.run as the core [08:20] lifeless: subunit.run doesn't do test discovery, does it? [08:20] wgrant: it does if you have python 2.7 or the discovery module installed [08:21] Ah, handy! [08:21] Thanks. [08:35] good morning [08:52] wgrant: So, that script doesn't love me. [08:59] StevenK: Oh? [08:59] FeatureError: Single aggregates aren't supported after a GROUP BY clause [09:00] bigjools: What do you think about replacing cron.daily-ppa with DiskPool improvements? [09:00] wgrant: I think we're going to have to shift back to ids ... [09:00] wgrant: see the oops-repository .testr.conf for inspiration [09:00] StevenK: Yeah, probably. [09:00] or [09:00] wrap the result as I suggested [09:00] wgrant: Will you rescind your issue with it if I do that? [09:01] see _getExternalthingy in translations for an example [09:01] StevenK: Maybe. [09:01] lifeless: No lp:oops-repository? [09:03] wgrant: https://code.launchpad.net/oops-repository [09:03] wgrant: sounds reasonable as long as it doesn't slow down the publisher [09:03] wgrant: I haven't setup a 'trunk' yet, nor all the metadata [09:03] figure out where empty dirs are left [09:04] bigjools: The file removal method will just check if the directory it's removing from is empty. [09:05] So it will slow p-d-r insignificantly. [09:06]