[00:01] wgrant: http://pastebin.ubuntu.com/1229519/ [00:01] StevenK: That looks most excellent. Perhaps try some tests that exercise the DB? [00:01] Hmmm [00:01] -t bugs ? [00:02] Though DatabaseLayer setup worked, so it's a good sign [00:02] And it can run while I nom [00:02] -t bugs is huge [00:02] Try archiveuploader [00:02] Relatively quick and terrible === matsubara is now known as matsubara-afk [00:37] wgrant: Right, that looks pretty excellent too [00:39] Right. [00:39] So, jenkins :) [00:41] wgrant: Yes [00:41] StevenK: Should be pretty easy, I guess [00:41] No need for the complex build rules any more [00:41] Just a .testr.conf, lp-setup-lxc-build, testr run --parallel [00:41] Or so [00:42] Do I need the testr from testing-cabal? [00:48] StevenK: You might need new testrepository, testtools and subunit [00:48] Might as well grab them all from there [00:48] Right [00:48] Just installing java and fixing ssh access [00:52] lol [00:53] The Jenkins slave is java, no choice in the matter [00:53] i know, just trolling [00:55] Slave successfully connected and online [00:55] wgrant: So, .testr.conf ? [00:55] This is different from the .testr.conf already in the LP tree? [00:56] StevenK: Yeah. Grab it from testr.conf in lp:~launchpad/lpbuildbot/public [00:56] It has a list command and uses lp-setup-lxc-test rather than bin/test [00:57] So I should move it into the tree as the first step? [00:57] Just clobber the .testr.conf that's in trunk locally for now [00:58] buildbot copies it in at the start of each build [00:59] I've added a build step [01:01] Project devel build #1091: STILL FAILING in 1 min 3 sec: https://lpci.wedontsleep.org/job/devel/1091/ [01:01] Heh [01:02] sudo lp-setup-lxc-build lptests /home/jenkins/.ssh/id_rsa [01:02] sudo: lp-setup-lxc-build: command not found [01:02] :-( [01:02] Is it in PATH? [01:03] It's in /home/jenkins/bin, but it looks like that doesn't work [01:03] ~/bin isn't in PATH by default [01:03] Well, yes [01:04] Project devel build #1092: STILL FAILING in 8.9 sec: https://lpci.wedontsleep.org/job/devel/1092/ [01:06] Project devel build #1093: STILL FAILING in 28 sec: https://lpci.wedontsleep.org/job/devel/1093/ [01:06] Haha [01:06] Need testr init first [01:06] Ooh [01:06] It nearly worked [01:06] Are you spying? :-) [01:06] Yes [01:07] How many threads does the instance have? [01:07] * StevenK firewalls wgrant off [01:07] 4 processors, apparently [01:07] You'll want testr run --parallel --concurrency=SOMETHING --subunit --full-results [01:07] And you'll need to set a shared TEMP, which probably means it needs to be in the code dir [01:08] buildbot creates temp/ in the tree at the start of each build [01:10] Oooh [01:10] This is looking okay [01:10] Indeed [01:10] concurrency=8 is probably going to die [01:10] But we'll see [01:11] Die how? [01:11] We have 34GiB of RAM [01:11] It'll thrash [01:11] Ah, so at least RAM won't be a problem [01:11] Oh, yeah, java. [01:11] Also, your build failed [01:11] So we have 4GiB of RAM available [01:11] lazr.restful missing from download-cache? [01:11] buildbot's happy with it, so I assume you just failed to update [01:11] Project devel build #1094: STILL FAILING in 2 min 19 sec: https://lpci.wedontsleep.org/job/devel/1094/ [01:12] Yeah [01:12] But it otherwise worked [01:12] I guess I need to run bzr pull in the download-cache [01:12] You'll want to add bzr up download-cache, and probably update-sourcecode, steps [01:12] But you can just do a one-off for now, I suppose [01:13] I don't need db setup? [01:13] lp-setup-lxc-build should do that [01:13] I thought so [01:14] Here's a script I prepared earlier [01:14] Project devel build #1095: STILL FAILING in 10 sec: https://lpci.wedontsleep.org/job/devel/1095/ [01:14] Evidently not :) [01:14] StevenK: scripts/init_testr.py from the lpbuildbot branch may be handy [01:15] StevenK: It preserves .testrepository between runs, which makes balancing the tests between runners more effective [01:15] And prevents the issue that killed this build :) [01:16] Hmmm [01:16] Right [01:17] So far I'm blowing it away, but that can be fixed [01:18] This is looking better [01:18] Indeed [01:19] wallyworld: Are you watching too, or just ignoring us? [01:19] huh? [01:19] Doing proper work, hopefully :P [01:20] This is proper work! [01:20] watching what? [01:20] Also known as buildbot and buildbot-poll die in a large fire [01:20] Jenkins [01:20] StevenK: Check in temp/ [01:21] Are there 8 files with lots of tests listed? [01:21] Yes [01:21] It's starting up 8 new containers, so it looks good [01:21] Great [01:21] I think we're good :) [01:21] Yay [01:21] We are [01:22] And it's even tagged properly [01:22] jenkins@domU-12-31-39-17-3C-7F:~/launchpad/lp-branches/workspace/devel$ uptime [01:22] I must work that out on prod buildbot with gnuoy next week [01:22] 01:22:17 up 3:08, 1 user, load average: 6.70, 2.75, 1.14 [01:22] :) [01:22] wgrant: i recall reading somewhere why the test count is inflated now. can you remind me? [01:23] wallyworld: The Zope testrunner layer setup/teardowns appear as tests [01:23] eg 'test: lp.testing.layers.BaseLayer:tearDown' [01:23] really? [01:23] I wonder how it will take with 8 [01:23] surely we can filter that out [01:23] If you grep them out then the count is accurate [01:24] StevenK: Didn't jenkins previously show live test stats? [01:24] It will show progress, not stats [01:25] But it takes like 3 or 4 builds to prime [01:25] :( [01:25] Yeah [01:25] Doesn't subunit2junitxml let it show live counts? [01:25] ENOIDEA [01:27] wgrant: StevenK: if you want a break from jenkins, a quickie.... https://code.launchpad.net/~wallyworld/launchpad/embargoed-sharing-policy-ui-1055617/+merge/126585 [01:27] 8 threads is probably what, 2 hours? [01:27] wallyworld: It's building [01:28] So now wgrant and I should do a critical or something [01:28] do a review first :-) [01:28] I'm waiting for a local testr run too [01:29] oh bollocks. legit bb failure [01:29] * wallyworld fixes [01:29] Line 53 can be sucked back onto 47? [01:29] Which probably makes the branch neutral [01:29] yeah [01:30] jenkins@domU-12-31-39-17-3C-7F:~/launchpad/lp-branches/workspace/devel$ uptime [01:30] 01:30:10 up 3:16, 1 user, load average: 11.62, 9.77, 5.15 [01:30] StevenK: Well, you did ask it to run 8 multiprocess test suites on 4 cores :) [01:31] I should have gone bigger? [01:31] Doesn't matter too much [01:31] Pft, we have 20GiB of RAM free [01:32] I'm not sure what yellow used for scalability testing, but we don't really need that here [01:32] We don't care about scaling, just working [01:33] It seems to work [01:33] Indeed [01:34] StevenK: You should probably steal the remaining build steps from master.cfg in the lpbuildbot branch [01:34] With scripts/init_testr and co [01:34] Aww, can't we just move on and get this on prase? :-P [01:34] Also shake Julian until MAAS Tarmac and Jenkins configs fall out! [01:35] I think other projects have used it as well [01:35] openstack is one I can remember. Until they switched to git, at least [01:52] StevenK: you ok to +1 that branch you looked at? [01:53] wallyworld: Sorry, done. [01:53] np thank you [01:54] i was doing the testfix anyway till now [02:24] wgrant: pqm 0.6 up on LP, and in trunk. [02:25] wgrant: YMMV, may the farce be with U [02:26] Thanks [02:26] Of course I might just dump PQM and use Tarmac instead :) [02:26] fine by me [02:26] long as you're not running tests under it [02:27] tarmac isn't even faintly paranoid enough about *that* [02:27] Tarmac+Jenkins ? [02:27] Right, Tarmac + Jenkins is the sensible solution [02:27] though I don't know why you'd have tarmac in the picture for that. [02:28] Other projects (eg. maas) have you submitting to Tarmac, Tarmac asks Jenkins to build it, then Tarmac commits if Jenkins succeeds [02:29] you may not know this, but PQM can get its queue from LP [02:30] or could before the code team destroy the API :) [02:30] anyhow [02:30] there is something funky with that layering [02:30] I need to reset my whiteboard to make a case, i think [02:33] do we know why TestArchiveWebservice.test_getAllPermissions_constant_query_count sometimes fails? [02:34] wallyworld: It might depend on whether it's the first browser test in that process [02:35] lifeless: Yeah, but Tarmac is at least less crufty and abandoned than PQM is today... [02:39] tarmac + jenkins is working extremely well for maas so far [02:40] bigjools: Are the configs available so we can duplicate it? [02:50] wallyworld: You fail. Again. [02:50] huh? [02:51] how? [02:51] Oh, the failure wasn't your fault. [02:51] nope [02:51] Carry on [02:54] Hm [02:54] That subunit stream looks a little broken [02:54] wgrant: I wonder if bug 1020439 is limited to those few branches [02:54] <_mup_> Bug #1020439: AttributeError: 'NoneType' object has no attribute 'items' in +preview-diff < https://launchpad.net/bugs/1020439 > [02:54] StevenK: Yes [02:54] Oh, they have no merge_diff ? [02:54] So assign to you and close [02:54] Oh oops, wrong bug [02:55] * wgrant -> lunch [02:55] Haha [02:55] Helpful much [02:55] It may be the same, but I can't remember [02:55] And am on my way out [02:55] wgrant: It still happens [02:55] wget a URL from the OOPS gives 00 [02:55] *500 [02:58] The MP exists, and has a diff [03:01] Project devel build #1096: STILL FAILING in 1 hr 46 min: https://lpci.wedontsleep.org/job/devel/1096/ [03:02] Wha? [03:02] * StevenK prods Jenkins [03:04] Hm, it misreports the failure, too :-( [03:39] * wallyworld has an appointment with the tax man :-( [03:39] wallyworld: To pay him? [03:41] no, to figure out some of my shit for last year [03:41] bbiab [03:56] Don't you pay him for that? :) [03:57] We don't pay him for his shit. [03:57] That would just be wrong wrong wrong [03:58] Heh [04:48] wgrant: Halp? [04:49] wgrant: bug 1020439 is triggered when previewdiff.diff.diffstat = None and I'm trying to figure out how to not make lazr.restful eat a bullet [04:49] <_mup_> Bug #1020439: AttributeError: 'NoneType' object has no attribute 'items' in +preview-diff < https://launchpad.net/bugs/1020439 > [04:52] StevenK: does lazr.restful want diff to never be None ? [04:52] lifeless: Where is that defined? [04:53] the schema for the interface [04:53] I'm wagging here, of course [04:53] Project devel build #1097: STILL FAILING in 1 hr 46 min: https://lpci.wedontsleep.org/job/devel/1097/ [04:53] Jenkins, you make me very sad. [04:53] lifeless: readonly=True, I'm not sure what the other defaults are [04:54] me neither [04:54] They are not set to required=True [04:54] So I guess they're False [04:56] _StringException: lost connection during test 'lp/registry/javascript/tests/test_milestone_creation' [04:56] Hm [04:56] StevenK: so - marshaller [04:57] StevenK: look at the marshallers available, I'll bet that its one with a bug in it [04:58] [05:00] StevenK: so a few lines up [05:00] marshaller = getMultiAdapter((field, self.request), IFieldMarshaller) [05:00] that suggests that that is going off the rails, to me at least [05:01] lifeless: How? [05:01] The field is declared as an exported dict [05:01] And it's attempting to unmarshall the contents [05:01] It just doesn't deal with None at all [05:01] right [05:03] Right, if value is None: return {} works nicely [05:03] HAH [05:03] def _get_diffstat(self): [05:03] if self._diffstat is None: [05:03] return None [05:04] GIGO [05:04] EWTF [05:04] how is that different to [05:04] return self._diffstat ? [05:04] Below that it returns a dict [05:04] ah [05:08] Fantastic, I don't need to touch lazr.restful [05:12] lifeless: So, can you look at the last Jenkins build? [05:12] lifeless: I have this inkling there is something screwy with the subunit stream [05:13] url me up [05:13] lifeless: https://lpci.wedontsleep.org/job/devel/lastBuild/ [05:18] ERROR: cannot verify lpci.wedontsleep.org's certificate, issued by `/C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Class 1 Primary Intermediate Server CA': [05:18] Issued certificate has expired. [05:21] Yes [05:21] We should fix that by moving to prase [05:21] * StevenK bangs that drum a little harder [05:23] StevenK: ok, so tricks: [05:23] cat consoleText| subunit-stats | less [05:23] page down till you see [05:23] Starting up the container... [05:23] after that you should see nothing as subunit will be eating it [05:24] when you start seeing something, its because subunit thinks a test prior to it has not finished [05:24] and is in pass-through mode on everything other than a terminal for that test. [05:24] so [05:24] test: lib/lp/soyuz/stories/ppa/xx-private-ppas.txt [05:24] is the first output [05:24] use that to seek in the raw stream to debug [05:24] less consoleText [05:24] / lib/lp/soyuz/stories/ppa/xx-private-ppas.txt [05:24] and work back: [05:25] test: Could not communicate with subprocess [05:25] time: 2012-09-27 04:39:15.455646Z [05:25] test: lib/lp/soyuz/stories/ppa/xx-private-ppas.txt [05:25] ^ thats invalid yo [05:26] Could not communicate with subprocess is zope.testing garbage, isn't it? [05:26] yes, its odd that it says 'test: Could not communicate with subprocess' [05:27] theres no immediate evidence of interleaving going on [05:27] it *could* be a race condition with gc or something [05:27] or just bad code for handling that situation [05:27] e.g. [05:27] there is a reporter thingy in zope.testing [05:27] it has a special class for subunit [05:27] that class may have a bug [05:28] where it reports this situation via the literal 'test: Could not communicate with subprocess\n' [05:28] which would be a bug. [05:28] Then why doesn't buildbot see that :-( [05:28] it may not be encountering 'Could not communicate with subprocess' [05:28] or it may be one of the things gary keeps posting a bug link to [05:29] StevenK: s def error_with_banner(self, message): [05:30] self._emit_fake_test(message, self.TAG_ERROR_WITH_BANNER) [05:30] Haha [05:31] That does seem a little bit like crack, yes. [05:31] which calls startTest(test), _emit_tag(tag), addSuccess(test) [05:31] so, that should be ok, but.. [05:32] and in fact.. [05:32] test: Could not communicate with subprocess [05:32] is line 108890 [05:32] successful: Could not communicate with subprocess [05:32] is 108996 [05:33] subunit doesn't like tests with whitespace? [05:33] so there is an interleaving problem [05:33] subunit doesn't care about that [05:33] open consoleText with vim [05:34] something wrote, to the same fd, a tonne of other test info between the three lines of that self._emit_fake_test function [05:34] wallyworld, wgrant: https://code.launchpad.net/~stevenk/launchpad/preview-diff-none-type/+merge/126601 [05:35] lifeless: Yeah, the line numbers are staggeringly different [05:36] StevenK: part of the issue here is that you're getting the demultiplexed stream out of testr [05:36] testr run --parallel --subunit --full-results --concurrency=8 | subunit2junitxml -f -o test-results.xml [05:37] yeah [05:37] testr is running 8 streams on separate fd's and demultiplexing them. [05:37] its possibly a bug in your testtools version [05:37] what testtools and subunit versions are you running ? [05:38] ii python-testtools 0.9.14+bzr267~ppa37~precise1 Extensions to the Python unittest library [05:38] ii subunit 0.0.8+bzr176~ppa126~precise1 command line tools for processing Subunit streams [05:38] Both from testing-cabal [05:39] hmmm [05:39] I have to run, cynthia time [05:39] suggestion [05:39] drop the subunit2junitxml -f -o test-results.xml for now [05:39] 54 [05:39] and the --full-results and the --subunit [05:39] argh, sorry! [05:40] testr will write its output to .testrepository, the jenkins fs explorer thingy will let us pull the stream down and see what it gets [05:40] I will return in ~2 hours [05:40] Which is how long the build will take [06:18] StevenK: Did the build look successful other than the dead runner and corrupt stream? [06:19] wgrant: Mostly [06:19] I'm doing what lifeless' suggested, so we'll see [06:20] But since testr likes hiding things, we have no output at all [06:20] StevenK: Tail the stream [06:20] It's in ~/.testrepository [06:20] It'll be tmpFOO [06:21] Right [06:21] wgrant: Can haz review? [06:21] Er [06:21] Not ~/, but the tree [06:22] Yes, I found it [06:22] -rw------- 1 jenkins jenkins 3.4M Sep 27 06:22 .testrepository/tmpTDk8xw [06:23] StevenK: Nothing else relies on diffstat being None sometimes? [06:24] Hm, maybe [06:28] wgrant: Good catch. http://pastebin.ubuntu.com/1229847/ [06:28] Well [06:28] This is interesting [06:29] I think our bugtracker may be sentient [06:29] https://bugs.launchpad.net/launchpad?field.searchtext=quote_like [06:29] Look at the first result [06:30] First thought was "these results are terrible". Open first result, "Cannot search for identifier containing underscores... well I can search for it but the results aren't any good." [06:30] Haha [06:31] wgrant: I can't see any other code that relies on diffstat being None [06:31] Did you consider making lazr.restful deal with it? [06:32] I did, and then decided it was a case of GIGO [06:32] Well, we allow object references to be nullable [06:32] Why not dicts? [06:34] Like you say, it could be fixed in lazr.restful's marshaller. But I decided that fixing LP was a little easier [06:35] And it didn't involve working out how to run its tests and then spinning a new release, uploading to pypi and download-cache and then making an LP branch anyway [06:37] If you strongly object then I can do that on Tuesday [06:39] StevenK: It's not that hard; wallyworld did it yesterday [06:40] and finished the job today [06:40] I know it's not hard. It's easier to just change LP. [07:06] wallyworld__: I don't think lp:lazr.restful contains your changes [07:06] hmm. i'm sure i merged, let me check [07:07] Hmm, yeah, revision 200 [07:07] ah, merged restfulclient [07:07] forgot restful [07:07] let me do it now [07:08] The version.txt in the tree and pypi did not match [07:08] And I didn't want to unfix your carefully fixed bug [07:09] StevenK: done, sorry [07:15] https://code.launchpad.net/~stevenk/lazr.restful/dict-unmarshaller-none/+merge/126615 [07:16] StevenK: Why isn't None None? [07:16] None shouldn't be {} [07:16] It should be None [07:16] probably [07:16] Well, it's better than what we have currently, which is AttributeError [07:17] Exposing an unstable interface in lazr.restful is not better than AttributeError :) [07:19] wgrant: StevenK: i had to reboot again after another unity crash and now lp.dev won't load in ff, but loads in chrome. "The connection was interrupted". any idea? [07:20] Stop stabbing zope while loading it in Firefox? [07:20] i have no knife [07:20] wallyworld__: Tried restarting Apache? That sometimes indicates it's SSL listening on a non-SSL port or vice-versa [07:20] wgrant: yeah, tried that [07:21] wgrant: Diff updated [07:22] StevenK: Have you tried this with lazr.restfulclient? [07:22] * StevenK de-prams his own toys [07:23] wgrant: No, I haven't [07:27] Project devel build #1098: STILL FAILING in 1 hr 45 min: https://lpci.wedontsleep.org/job/devel/1098/ [07:28] That isn't fun. [07:52] good morning === rvba is now known as Guest78352 === almaisan-away is now known as al-maisan === _rvba is now known as rvba === al-maisan is now known as almaisan-away === gary_poster|away is now known as gary_poster === matsubara-afk is now known as matsubara [13:17] Say... I don't have a working launchpad setup at the moment. Would anyone be able to test & land this branch for me? https://code.launchpad.net/~jtv/launchpad/format-relative-imports/+merge/124878 === Guest32361 is now known as slank === slank is now known as Guest2828 [13:53] jtv: sending via ec2 [13:53] ? [13:53] Oh, branch. Thanks! [13:53] responsive r us [14:04] stub: talking of responsiveness, have a moment to eyeball https://code.launchpad.net/~stefanor/launchpad/edit-packagesets/+merge/124555 ? [14:12] tumbleweed: done [14:13] stub: thanks === jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: jcsackett | Firefighting: - | Critical bugs: ~300 === slank is now known as Guest55818 [16:29] jcsackett: thanks for your review! [16:29] * adeuring nearly forgot the MP [16:30] adeuring: you're welcome. === matsubara is now known as matsubara-lunch === matsubara-lunch is now known as matsubara [18:28] sinzui: https://bugs.launchpad.net/launchpad/+bug/736010 is affecting openstack [18:28] <_mup_> Bug #736010: Milestone:+index timeouts < https://launchpad.net/bugs/736010 > [18:29] sinzui: wondering if one of your timeout demolishing machines could cast an eye over it. [18:30] deryck: I'm getting a test failure on stable. Both test and code are old. Can you reproduce? http://pastebin.ubuntu.com/1230926/ [18:32] abentley, let me see.... [18:35] sinzui: oh, I see ttx is in #launchpad talking abou tit [18:42] lifeless, We have discussed that bug a lot. We think a lot of work is needed t fix this issue. This is a design problem, not a a bad query problem [18:44] abentley, I don't get that failure. [18:44] sinzui: Why do you say its a design problem ? [18:45] sinzui: I see 2000ms of time between query 66 and 67 in https://oops.canonical.com/oops/?oopsid=OOPS-c3db3abe60cbad08100ecb38fcac389a#statementlog [18:45] deryck: Wacky. It looks legit to me, but it also doesn't seem to happen on ec2. [18:46] WEIRD [18:46] sorry to shout, fat fingers :) [18:46] I wonder what's different for you that you see it? [18:47] lifeless, the page only works if you can display all milestones and blueprints. It does not scale. We can improve the queries, but it still will not scale for large projects. I think a proper fix is to load just the crucial/first items, then use ajax to complete the listings, then do the analysis. [18:47] deryck: Maybe python version. Are you on quantal 2.7.3? [18:47] sinzui: I am confused; the OOPS doesn't show queries as a problem at all. [18:47] sinzui: Are we talking about the same issue ? [18:48] abentley, ah, no. I still have upgraded actually. :( [18:48] abentley, but I am on 2.7.3 [18:48] yes we are, This bug has been around a long time and making queries faster does not fix the issue [18:48] sinzui: ok, so we're slightly in sync :) - I see bad python code / tal code. [18:49] You can fix that query, but it want users the believe you have fixed the issue for there hundreds of items, we need to think about the solution differently [18:49] YES [18:49] sinzui: I agree that doing latency hiding and incremental loading would be better. [18:49] Late evaulation that because the page relies on bug and blueprint rendering rules [18:49] sinzui: but is there not a low hanging just-be-more-efficient step to get Ubuntu and openstack unstuck ? [18:49] sinzui: its not late evaluation, only 68 queries on the page before it timed out . [18:50] I think we just want to pull the data and have the browser do the analysis...such as I do in the burndown charts I wrote [18:50] sinzui: those charts are sweet ;) [18:51] I don't believe the issue is low hanging. I have landed more than a dozen speed ups in 2 years. so have you and William [18:51] I asks jcsackett to not leap into that bug given the tremendous failure rate [18:52] hmm [18:52] loaded for me [18:52] lifeless, sinzui: timeout fixed. [18:52] 69 queries in 6.83 seconds [18:52] jcsackett: how ? [18:52] as i said, was increasing the wrong hard_timeout rule. [18:52] * jcsackett misread the flags. [18:52] the flag was for ProjectMilestone; we just now fixed milestone [18:52] well "fixed" ;-p [18:53] lifeless, wgrant, jcsackett, and wallyworld__can discuss the effort to make that page properly fast for all project sizes in a few hours [18:54] sinzui: my curiosity is piqued [18:54] sinzui: I agree on the long term [18:54] but I'm wondering why the short term is so poor [18:54] certainty. [18:55] 1 day of effort to fix one bug is high certainty to deliver value [18:55] 5 days to fix 5 bugs is less certain, but equal value [18:56] 10 days to fix 1 bug is even less certain to deliver less value. [18:57] lifeless, we are a "machine" because we are taking the most certain bugs first. I asked jcsackett to not work on that one yet because there is more risk involved. [18:57] * sinzui hoped to fix another escalated bug before having to do feature analysis to fix criticals [18:57] sinzui: all cool [18:58] sinzui: I meant, what about the code makes this one so poor [18:58] sinzui: not about your team or your choices ;) [19:00] we often choose to make pages do less, or batch. milestones do need to show all items. The tales calls are an embarrassment. I choose to reuse code to keep presentation consistent, but the adaptions are costly. [19:02] lifeless, the analysis is costly in Python I think. I know I can do it in JS now. [19:05] sinzui: I'm getting a profile [19:06] https://qastaging.launchpad.net/++profile++show/ubuntu/+milestone/ubuntu-10.10/+index# [19:06] just sneaked in: 3378720 function calls (3296251 primitive calls) in 19.845 CPU seconds [19:12] sinzui: there are some low hanging fruit that might make a lot of things better (or might be profile artifacts) [19:12] 5134 0.810 0.000 0.979 0.000 enumcol.py:39(__init__) [19:12] ^ 4% of total runtime, I believe wgrant has complained about enumcol performance before [19:12] yes [19:13] a few more like that [19:14] this page might break now that there are private bluerprints too. milestones do the private bugs query then cache the permissions, but I did not see such as change land for blueprints [19:15] yay [19:19] sinzui: the ubuntu timing out page is only 140k in size [20:24] jcsackett: I have a sequence of four branches that need review. Are you up for it? [20:24] abentley: if said sequence starts with new-tests, i'm already looking at it. [20:25] jcsackett: Excellent. Continues with https://code.launchpad.net/~abentley/launchpad/storm-sprint-queries/+merge/126770 https://code.launchpad.net/~abentley/launchpad/specification-cleanup/+merge/126789 https://code.launchpad.net/~abentley/launchpad/hide-sprint-blueprints/+merge/126792 [20:27] abentley: cool. i can get 'em all. [21:05] jcsackett, do you have time to review https://code.launchpad.net/~sinzui/launchpad/email-authentication-errors/+merge/126801 [21:06] sinzui: sure, you're in the queue. [21:06] still looking through abentley's series. [21:12] abentley, rick_h_. was the privacy banner or information portlets updated in the last few days [21:12] JS is broken on pages that show the privacy banne like bugs and archives [21:13] sinzui: Yes, rick_h_ changed the banner code yesterday IIRC. [21:13] fab, maybe the diff will help me find the problem [21:14] sinzui: (rick_h is away) [21:14] okay, diff it will be [21:35] abentley: i'm through your series of branches. r=me on all but one which has a few questions. [21:35] jcsackett: I've replied. [21:35] jcsackett: Thanks! [21:38] abentley: missed both things you pointed out. given those, r=me on that one as well. [21:38] * jcsackett moves on to sinzui's. === jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: ~300 [21:39] wallyworld__, If your waking: read this to learn what I have learned so far https://bugs.launchpad.net/launchpad/+bug/1057714 [21:39] <_mup_> Bug #1057714: javascript is broken on pages that show the privacy banner < https://launchpad.net/bugs/1057714 > [21:40] sinzui: haven't read that bug but i think it may be a dupe of mine from yesterday, for the mp ypu approved [21:40] mine is bug 1057248 [21:40] <_mup_> Bug #1057248: javascript error on bugtask page < https://launchpad.net/bugs/1057248 > [21:40] wallyworld__, does this look familiar: http://pastebin.ubuntu.com/1231240/ [21:41] This is the change, but I do not know why info_type is not defined in the privacy module [21:41] sinzui: yes. i changed privacy.js and information_type.js [21:41] okay, [21:41] there were vrious issues - 'throw' was misspelt, lots of lint [21:42] ah@ [21:42] plus the info_type issue [21:42] ! [21:42] okay [21:42] landing my branch now [21:42] wgrant: enumcol would benefit from a cache I suspect [21:42] wgrant: one of the rare times I'll say that [21:42] we can deploy this morning [21:42] we can run the yui layer locally, then lp-land [21:42] sinzui: yes, doing that now [21:44] sorry wallyworld__I remember the diff, but I did not read all the details int he bug. I suck [21:45] sinzui: r=me on your branch. [21:45] wallyworld__, I was rushing to review branches because I had a day trip to look at houses [21:45] thank you jcsackett [21:52] np [22:54] wgrant: We've flushed nearly all the queue now following beta-2, and it all seemed pretty snappy; no timeouts in 80+ accepts. At this point I'm confident with going ahead and removing the queue script. Is there any remaining review you feel the need to do on https://code.launchpad.net/~cjwatson/launchpad/remove-queue-tool/+merge/114464 ? [23:01] cjwatson: Let me look [23:03] lifeless: Hmm [23:03] lifeless: I might try to profile it a bit harder now that someone other than me thinks it might be a problem. [23:05] wallyworld__: man, so sorry. Just thought I added unused stuff. Tests passed, etc. I shouldn't have tried to rush that before I left [23:06] cjwatson: r=me. Are you also going to remove the feature flag check? [23:07] rick_h_: no problem. shit happens :-) [23:08] wallyworld__: I owe you one. broke it apart and tried to stage it in nice simple bits that didn't get too involved and boom [23:09] rick_h_: we may need to look at the events - seems there are 2 sets (show/hide banner, info type public/private) that do the same thing. may be able to be refactored to just one set [23:10] wallyworld__: well so I can explain that [23:10] i didn't get into it too far [23:10] wgrant: That's going through EC2 at the moment (https://code.launchpad.net/~cjwatson/launchpad/pabj-remove-feature-flag/+merge/126090) [23:10] because the filbug.js uses the privacy banner directly with a custom message, you need to be able to talk to the privacy banner with a custom message [23:10] cjwatson: Ah, great [23:10] I have a fair bit of MP mail; haven't quite caught up this morning yet [23:11] Yep. Thanks. I look forward to my giant LoC credit [23:11] wallyworld__: but the normal use case is that someone changes the information_type widget, which fires the information_type:changed, and files information_type:is_public/private and the banner will auto show/hide based on that event [23:11] And I haven't even removed delayed copies yet ... [23:11] wallyworld__: so there's really two use cases and almost should be a different banner, since privacy and security are sharing code tbh [23:11] wgrant: there is a kcachegrind file for you [23:11] rick_h_: ok, thanks for the explanation [23:11] my main aim was to add the missing tests and get the fix out [23:12] wallyworld__: and in the next branch the banners will render themselves vs the html being in the .pt file, so the events will be used as ways to say "need a banner html setup here" [23:12] cool :-) [23:12] wallyworld__: right, understand. and it's not 100% tested because it's part 1 of 4 and in general this was 'new unused' code since nothing is firing the events yet [23:13] rick_h_: one thing. i had to move the info_type namespace variable from the top of the module or else it was undefined [23:13] i don't know why [23:13] wallyworld__: so like I said appreciate it. Been offline most of the day on vaca so bummed to log in and see I did wrong for you guys [23:13] wallyworld__: hmmm, let me look [23:14] lifeless: But it's from qastaging, so it's not completely useful [23:14] Not useless, though [23:14] not useless [23:15] wallyworld__: ok that's strange. I'll check it out when I get to the follow up branch when I get back [23:15] rick_h_: Missing context here, but we only recently moved the banners *into* the HTML, to avoid them popping into the page later on once the JS loads. [23:15] rick_h_: thanks. i didn't get into the root cause. i'd love to know why [23:15] wallyworld__: nothing looks odd, but running on 3hrs sleep after driving 600+ miles overnight [23:16] rick_h_: where did you drive from/to? [23:16] wallyworld__: https://maps.google.com/maps?saddr=clarkston,+mi&daddr=Charlottesville,+VA&hl=en&sll=37.6,-95.665&sspn=38.593229,86.396484&geocode=FRAWjAIdYh8H-ymTC47xX5ckiDGKYHxzNWKthw%3BFfpHRAIdeopS-ymPpFDqLYaziTH8dIvDlvCGkA&oq=charlottesvi&mra=ls&t=m&z=6 [23:16] doh [23:17] looks like a nice drive [23:17] wgrant: yea, I'll hopefully be able to take care of that still. The big thing is we need banners on more places and it takes a bunch of code to wire it up it seems. Trying to split it into contained parts [23:18] wgrant: for instance trying to not have the privacy banner need to know about the information type, LP.cache locations, etc. [23:18] Indeed. [23:18] wgrant: but thanks for the heads up. I didn't realize it was moved due to a flash effect so good to keep an eye on [23:19] wallyworld__: yea, well with a 2yr old those overnight drives still the best so he sleeps through most of it. Thanksfully we also had the lion king movie soundtrack for backup lol [23:19] ah, those were the days. long gone for me now [23:19] rick_h_: wgrant still listens to the Lion King for his beddy byes time [23:19] Heh [23:20] hah [23:21] wgrant: the other thing is there is a huge amount of content being re-rendered [23:22] wgrant: a simple static result cache for rendering of person priority importance status might help a great deal. [23:22] We can possibly just also make that not terrible [23:22] or do both [23:23] evaluating the same template with the same parameters in the same request to the same bytes seems non horrible to me [23:23] Says he who removed template fragment caching :P [23:23] (yes, yes, it was the wrong way to cache, but blah) [23:23] :) [23:24] I'm not against caches /per se/ [23:57] wallyworld__: qas is updated, if you want to check out the JS stuff [23:57] Not sure why it's [testfix] [23:57] finally, been waiting [23:57] wgrant: because db bb was borked and i needed to get it landed [23:57] we should fix that [23:58] wallyworld__: In that situation you should force the build and wait 5 minutes [23:58] [testfix] skips all QA [23:58] ok