[00:44] 13 cards in QA-Landing, because buildbot sucks and needs to die. === StevenK changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: -ey | Firefighting: - | Critical bugtasks: 4*10 === StevenK changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer-ey | Firefighting: - | Critical bugtasks: 4*10 [00:45] * StevenK stabs irssi [00:45] StevenK: ping, it came up in a call that you and Steven hit a bug where codehosting was giving bad responses during log rotation? [00:45] StevenK: did that end up in a bug somewhere that I can reference? [00:45] That I and me? [00:45] :-) [00:45] :) [00:46] just going off of what I'm told from the higher power, Steven and Steve K were involved somehow [00:46] It was investigated by poolie, wgrant and I. [00:46] Only one of us is called 'Steve' [00:46] ok, well then I was lied to, my apologies [00:46] https://bugs.launchpad.net/bugs/724750 [00:46] <_mup_> Bug #724750: launchpad redirects pack files to https://launchpad.net (while repacking?) < https://launchpad.net/bugs/724750 > [00:47] That's the one. [00:47] ty sir [00:47] It doesn't impact buildbot since that uses bzr+ssh. [00:47] rick_h: BTW, thanks for fixing the YUI2 thing. [00:47] wgrant: sorry it got hung up with my travels and such [00:48] had hoped to get it landed before I left, but various tests hated me [00:48] Yeah, that often happens :( [00:48] Now maybe I can implement Content-Security-Policy this weekend. [00:49] rick_h: You've recovered from PyCon? [00:49] Still, 30 minute test suite soon :) [00:49] StevenK: working on it, have the post pycon flu so on meds and need a nap something fierce [00:50] wgrant: yea, heard there was some serious progresson that, exciting! [00:53] wgrant: So should I replace bug.private with bug.information_type in (2, 4, 5) in the triggers? [00:54] StevenK: (3, 4, 5), but yes. [00:54] StevenK: That's what I've done, although it is a bit awful. [00:54] Yup [00:55] wgrant: The DB branch that makes information_type not null failed due to the garbo job. :-( [00:55] Heh. [00:55] The garbo job has just been killed in devel [00:55] So I aw. [00:55] s [00:55] Now to wait for buildbot so I can re-land the db patch [00:56] That's about 11 hours away. [00:56] I'm aware. :-( [00:56] at now + 11 hours bzr lp-land [00:57] That has no hope of working, but it would be nice. [01:00] * StevenK claims another -12 patch [01:02] wgrant, wallyworld: I think we should suggest to sinzui that we need a new lane: 'QA-Fucked over by buildbot' [01:03] +1 :-) [01:03] Haha [01:03] Yeah [01:03] I'd really like there to be a separate column. [01:03] For when it is actually landed. [01:04] Since once it's landed it can usually be dealt with by anyone. [01:05] To be fair, QA-Landing is conflating ec2 with buildbot. [01:05] Perhaps we need two: "This is working through ec2, and isn't landed yet." and "This has hit devel/db-devel and can be dealt with by anyone when it hits qas." [01:07] That's what I meant, yes. [01:07] Right. [01:16] wgrant: CREATE FUNCTION changes to DROP/CREATE or CREATE OR REPLACE ? [01:18] StevenK: CREATE OR REPLACE [01:18] DROP will probably not work because there are references. [01:18] In general there's very little reason not to use CREATE OR REPLACE every time. [01:26] wallyworld: Hi, did you see my comment on your MP about half an hour ago? [01:27] wgrant: no. but i already fixed the typo [01:27] is person_sort_key case insensitive? [01:28] It is. [01:28] ffs [01:28] why would it do that? [01:28] Ah, hm, I see you did indeed fix the typo a few minutes ago, but I don't have an email about it :/ [01:28] Because case insensitive sorting is what most of the world expects. [01:29] luckily for the next landing it's not really an issue [01:30] regardless of what the world expects, it should match the case sensitivity of normal order bys [01:30] It's also rather trivial to fix without rerunning tests. [01:30] yes [01:30] Hm? [01:30] That *is* the normal order by. [01:30] Ordering by Person.displayname is not supported. [01:30] but Person.displayname, Person.name is [01:30] Its sort order is not a recognised one. [01:30] No [01:30] It's just a rule we have to break here until StormRangeFactory is fixed. [01:31] person_sort_key is, as the name suggests, the person sort key. [01:31] (Person.displayname, Person.name) is not, but it's close enough for the initial branch. [01:33] there should be no rule against sorting on multiple columns or am i missing something? [01:33] It's not a rule against it. [01:33] The rule is that the natural ordering for persons is person_sort_key. [01:33] Which is roughly lower(displayname) + ', ' + lower(name) [01:34] sure, ok [01:34] Having slightly different orderings around the place is rather confusing. [01:35] (Person.displayname, Person.name) will do for now, but the ordering for the batch slicing and intra-batch sorting must be identical, or it will be very confusing. [01:35] 18degC? What the heck. This isn't winter. [01:35] It got down to 11 here yesterday. [01:35] Bloody heck [01:35] http://www.baywx.com/melbtempyestII.html [01:35] Veeeery odd day. [01:36] isn't that graph basically the wrong way up? [01:36] Yes. [01:37] bloody freezing here too, it's 22C [01:37] wgrant: agreed about sorting needing to be consistent - i thought it was, didn't realise person_sort_key was case insensitive [01:38] wallyworld: Yep [01:38] bigjools: Yes, but you're in the past. === Ursinha is now known as Ursinha-away [01:58] wgrant: StormRangeFactory doesn't accept lower either [01:59] so we have to live with it till StormRangeFactory is enhanced [02:00] wallyworld: Sure, the only way to get it consistent now is to use plain (displayname, name) [02:00] which will mess up if there's names with non ascii chars [02:04] wallyworld: We don't really handle that case very well normally. [02:05] i'll tweak the sort order, raise a bug for StormRangeFactory, and put in an xxx [02:06] Thanks. [02:06] I'll probably end up fixing SRF soonish. [02:06] Since we likely want it for bugs. [02:09] wgrant: bug 961836 [02:09] <_mup_> Bug #961836: StormRangeFactory should support functional sort keys < https://launchpad.net/bugs/961836 > [02:31] wallyworld: Bug #961840 [02:31] <_mup_> Bug #961840: "Share with someone" makes it easy to accidentally revoke access < https://launchpad.net/bugs/961840 > [02:32] wgrant: currently "by design" since there's no vocab to support that sort of thing yet [02:32] wallyworld: k [02:33] wallyworld: Is it easy enough to use the normal batch spinner on initial page load? [02:33] but good that's it's filed so we can track it [02:33] Currently it uses a different one for initial vs batch load. [02:34] no since the batcher expects trhe initial batch to be rendered [02:34] server side [02:34] Yeah, it would obviously need some changes, but might be nice if it's simpleish. [02:34] i guess the client side code could be changed [02:34] Apart from that, this UI is pretty damn nice. [02:34] and faster than other things [02:56] lifeless: I'm seeing overhead of about 10% on +sharing from traceback collection from just 27 queries. [03:02] wgrant: what are the times [03:06] Sigh, libvte upgrade has broken termcaps [03:07] lifeless: Rendering without profiling is about 0.20s vs 0.22s. With profiling is 0.42s vs 0.48s, with about 0.05s shown in format_stack [03:08] But that's 2ms per query, which is sort of insane unless it's doing IO. [03:08] Ah, it is. [03:08] linecache.checkcache [03:08] I wonder how much of it is that. [03:10] wgrant: Can you link me to your branch that changes the triggers? [03:11] wgrant: so, I'm willing to pay 2ms on a 20ms call to have good data on the bad calls [03:11] wgrant: if you can come up with a sensible way to achieve the good data on bad calls without the 2ms on the 20ms call, I'd be delighted [03:11] lifeless: That adds 15ms to every authentication attempt :) [03:12] one thing we could do is get the line nos but not the line content [03:12] Yeah [03:12] and if we oops only then convert to lines [03:12] Exactly. [03:12] Since hopefully the file won't change much... [03:13] if it does we have a different problem [03:13] But I don't know how much is actually the formatting. [03:13] so our auth calls are 150ms? [03:13] We do have some pretty horrific traces due to TAL. [03:13] with 15ms in data gathering overhead? Still doesn't bother me :) [03:13] Request startup and auth is 50-60ms on production. [03:13] sorry, 165ms w/15ms in data gathering [03:14] wgrant: are you talking about the validation of credentials per request [03:14] Yes. [03:14] ok [03:14] so look, I'm totally happy with tweaks to make it better [03:15] I'm glad you're working on sufficiently lean stuff that this is showing up [03:15] It's not big, but it's not exactly insignificant as you initially stated :) [03:15] I think we're the only team that is following the directive that new pages need to be <1s [03:15] StevenK: https://code.launchpad.net/~wgrant/launchpad/legacy-access-type/+merge/98342 [03:15] StevenK: I think it's <5s [03:16] <1s is lifeless' eventually 99% goal, IIRC [03:16] the directive is for new pages to be <1 99% of the time [03:16] Ah [03:16] new stuff should be where we want to be [03:16] Oh right [03:16] Timeout of 5s. [03:16] right [03:16] could you please add feature rules to ensure that as you add page ids? that would be awesome. [03:16] The batching changes should bring this comfortably to 99% <0.5s, except for Ubuntu where who knows. [03:17] it will be brilliant to have front end problems [03:17] Indeed. [03:18] Would love a flaggable soft timeout... [03:18] doit [03:18] I tried. [03:18] Requires reworking lots of things. [03:18] oh yeah end of the world sort of thing [03:18] you could cheat [03:18] And it's probably worth taking it that little bit further to make UserRequestOops work in the same way. [03:19] At present ++oops++ doesn't work for requests that don't render the base template. [03:19] oh my, checkcache isn't written for speed is it. [03:19] Nooooo it is not. [03:19] so [03:19] you have my total blessing to monkey patch that to return. [03:19] we /don't/ support reload() in lp. [03:20] we could, in principle, use inotify but I don't see the practical value [03:21] maybe if you changed it to be iNotify, then the value becomes obvious. ??? [03:22] Heh [03:22] spm: keep your day job :) [03:23] 828 calls to checkcache from 27 queries [03:23] :'-( [03:23] And I guess each of those causes at least 1 stat [03:23] And those are all really shallow queries. [03:23] whats calling checkcache ? [03:23] getlines? [03:23] _get_frame_data [03:24] # End part adapted from zope.exceptions. [03:24] co = f.f_code [03:24] filename = co.co_filename [03:24] name = co.co_name [03:24] linecache.checkcache(filename) [03:24] line = linecache.getline(filename, lineno, f.f_globals) [03:24] Seems like at worst we'd want a clearcache at the start of the request. [03:24] its not threadsafe [03:25] or rather [03:25] I'll take money that there are check-and-then-reference patterns in the code [03:25] True. [03:25] which will blow up if the cache clears between two bytecodes in the model [03:25] module [03:25] like [03:25] ok [03:25] if filename in cache: [03:25] return cache[filename][2] [03:25] else: [03:25] return updatecache(filename, module_globals) [03:26] (in getlines, called from getline) [03:26] :D [03:26] now, if you want to rewrite the whole module more-or-less, sure. [03:26] the right place to check the cache is import [03:27] 'if you replace this module please be so kind as to discard its cached entries' [03:27] ditto tal, if tal uses the linecache to cache lines for tracebacks [03:28] and really, module -> lines should be a module held reference so that reload(),w hich leavces the old module to be gc'd, doesn't cause bad stack traces on old call stacks. [03:28] * lifeless stops before he gets critical [03:31] Before? [03:32] Mysterious. [03:33] * StevenK stabs MPs [03:33] Nothing is calling checkcache [03:33] But it's still being called. [03:33] zope is good at hiding behaviour [03:34] Oh [03:34] extract_tb does it [03:34] In stdlib's traceback module [03:34] Why were we calling it directly too :/ [03:34] To be certain. [03:34] Or something. [03:37] thats probably a bug [03:37] the intent was to have the oops core get tb's [03:37] but the lp consumer opt in for the zope pretties === StevenK changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10 [04:02] lifeless: A custom traceback collector without touching the files at all is 10x faster. [04:02] So <1% overhead. [04:03] Means we need to have a somewhat more hacked up traceback system, but it seems like it's probably worth it to do the formatting only on demand. [04:03] wgrant: I still get failures in TestBugMirrorAccessTriggers with your DB patch applied. :-( [04:05] wgrant: cool, if you're going to execute on this wicked, if not can you file bugs to capture the issue and your digging ? [04:06] StevenK: Do you also have my test fixes merged? [04:06] Oh, there's test fixes for TestBugMirrorAccessTriggers too? [04:06] yes [04:07] Best to merge the whole branch, probably. [04:07] I only grabbed the DB patch [04:09] And indeed, that was the only change in the branch you linked. [04:11] Oh, yes, different branch. [04:11] I was thinking of bugtaskflat. [04:11] Which had test changes. [04:11] What's the erorr? [04:14] wgrant: http://pastebin.ubuntu.com/894657/ [04:16] StevenK: Somehow the bug is acquiring an extra subscription. [04:16] In every case. [04:17] I wonder if that points to a bug in transistionToInformationType() [04:17] It may be implicitly subscribing the actor. [04:17] So if the actor's not the usual person, bad things happen. [04:18] Where "bad things happen" == "making this pedantic test fail" [04:18] But it's possibly good that it picked it up. [04:54] wgrant: Figured it out. [04:55] wgrant: I had changed createBug() to unilaterly subscribe the bug supervisor if the information_type was USERDATA instead of 'params.product and params.product.private_bugs' before I changed it. [05:01] StevenK: Ah [05:02] That would break apport. [05:02] No other tests failed? [05:02] s/break/leak/ [05:02] I was only running TestBug as a smoke test [05:02] Next up is -vvm bugs [05:05] lifeless: Bug #961875, and qastaging agrees with my numbers. 10% for the average well-optimised sub-second page. [05:05] <_mup_> Bug #961875: Timeline traceback collection has significant overhead < https://launchpad.net/bugs/961875 > [05:06] More like 5% for eg. +bugs [05:06] StevenK: So, my tests are happy now? [05:07] wgrant: Yeah, that fixed them. If I was doing -vvm bugs as my smoketest, I suspect I would have seen other fallout. [05:07] But -vvm bugs takes an hour [05:08] Yeah. [05:11] wgrant: TestBug == Total: 874 tests, 0 failures, 0 errors in 12 minutes 6.871 seconds. [05:15] Excellent. [05:20] wgrant: Running through -vvm bugs now, already at 27 failures [05:21] Heh [05:22] wgrant: which is ok :) [06:23] Hmm [06:23] We'll probably hit r15000 tomorrow. [06:26] wallyworld: Hum [06:26] wallyworld: The batching doesn't actually batch. [06:27] Ah, nevermind, the real batching rev isn't on qastaging yet. [06:27] Just the UI half. [06:27] 600ms in _load_object :/ [06:45] only a few more hours to go till it gets to qas [06:46] wallyworld: Still, most of the QA can be done now. [06:46] * wgrant QAs his stuff. [06:46] wgrant: which pillar were you looking at for the 600ms [06:47] wallyworld: launchpad [06:47] ++profile++show is handy [06:47] https://qastaging.launchpad.net/launchpad/+sharing/++profile++show [06:47] sure, i've been using it a lot this week [06:48] Actually, looks like 600ms was exception [06:48] 450ms is more common. [06:48] But asuka seems to be rather unstable :( === almaisan-away is now known as al-maisan [08:53] good morning [09:29] :( [09:30] hello launchpad do you know what is a good thing libraries that do not assume that a user is only running one program that uses that library at a time that is a good thing. [09:31] hello launchpad do you know what is a good thing libraries that do not assume that a user is only running one program that uses that library at a time that is a good thing. [09:31] sorry for the repeat [09:56] jml: Just disable the cache, I guess :/ [09:57] You can also pass a custom cache dir to the Launchpad constructor [10:10] jtv: good morning/evening [10:11] hi czajkowski [10:11] It's not the one I already commented on, is it? [10:11] nope [10:11] just reading that [10:11] thanks for looking at it [10:12] jtv: https://answers.launchpad.net/launchpad/+question/191340 [10:12] jtv: I am getting better at figuring the translations out, just I do seem to find unusal ones I need your help still [10:12] I'm a bit busy with a deadline, a review for rvba, and some SQL magic for dpm, and having network trouble so expect me to be a bit sporadic today. [10:13] The question page, for instance, isn't loading for me. This may take some time. [10:13] jtv: not urgent sorry wil try and find out [10:23] czajkowski: I just answered. === al-maisan is now known as almaisan-away [10:35] lol. "When complete", "A moment ago (estimated)" [10:37] jtv: thanks [10:37] czajkowski: if you do find time to look into this further, it's scripts/rosetta/remove-translations-by.py [10:38] It _may_ be possible to select something like “only by the given user.” [10:38] nods [10:40] czajkowski: (Yes, I know, I wrote that script. And come to think of it yes, you should be able to select the submitter. But the script also has some safety catches against mass deletions that you wouldn't want to do accidentally.) [10:42] wgrant: yeah, I passed a custom cache dir, but then I accidentally filled my (EC2 instance's) disk. [10:43] wgrant: I'm tempted to just fix lazr.restfulclient. [10:44] jml: Brave man. [10:47] well, let's see if I can actually run the tests. [10:48] I tried to do that once. [10:50] :( [10:50] If core developers can't patch it, then it's literally unmaintainable. [10:51] This was years ago. Not sure how it is now. [10:53] Then that just makes it literally unmaintained. :P [10:54] That is entirely uncontested. [11:03] danilos: ping === jtv is now known as mup === mup is now known as jtv [11:29] heh heh [11:30] so, a couple of weeks ago maybe, httplib2 decided that no one should be allowed to use 0.6.0 any more [11:34] I'm not sure why buildout is looking on the internet for this rather than in ./download-cache/dist [11:35] Oh [11:35] Does that mean that they admitted the certificate thing was actually a problem? [11:36] I honestly don't know. [11:37] Why isn't buildout fetching files from download-cache... [11:39] ah [11:39] it needs a setting in buildout.cfg that lazr.restfulclient doesn't have set. [11:50] zope.testing. there's no school like the old school. === Ursinha-away is now known as Ursinha === rick_h changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: rick_h* | Firefighting: - | Critical bugtasks: 4*10 [12:20] did something change with launchpadlib recenty? I'm getting unicode strings for datetimes now [12:21] rick_h: O hai. Can you please prod deryck to do his QA when you do that whole stand-up thing with him? [12:23] StevenK: email pls :) [12:24] mhall119: yes, there was an update to wadllib a bit back, I just ok'd a fix this morning [12:24] rick_h: Why? [12:25] StevenK: because then it's on his todo list and not something I can forget to bug him about during standup [12:25] rick_h: thanks, so it should be fixed in Precise soon? [12:26] mhall119: yes [12:26] I'd bug him directly, but he isn't on IRC. [12:26] rick_h: is there a bug # I can subscribe to about this? === rick_h_ is now known as rick_h [12:59] Morning, all. [13:00] deryck: O hai. Would you mind doing the QA for r14983? :-) [13:00] StevenK, sure, first thing on my list today. [13:15] czajkowski, Can you change the definition and implementation status https://blueprints.qastaging.launchpad.net/gdp/+spec/gdplaunchpad === almaisan-away is now known as al-maisan [13:15] woo-hoo, we now have the same number of queries issued in both old bug listings and new. [13:15] sinzui: nope [13:16] deryck: Woo! [13:16] sinzui: I checked all my old blueprints before I created the bug [13:16] as that's kinda annoying not to be able to change the status give this time of cycle [13:16] czajkowski, no yellow edit icons next to the two fields> [13:16] sinzui: nope [13:16] there used to be :) [13:16] czajkowski, the model says you can [13:16] * sinzui looks for old form [13:17] I can edit subscription [13:17] thats it [13:17] czajkowski, can you access and use https://blueprints.qastaging.launchpad.net/gdp/+spec/gdplaunchpad/+status [13:18] sinzui: yes [13:18] sinzui: how did you get there? [13:18] anyone can change subscription. I think the permissions on the links are different from the real fields [13:19] czajkowski, I help my pointer over the edit Icon I could see to learn the non-javascript location [13:19] sinzui: only edit filed I cna get to is https://blueprints.qastaging.launchpad.net/gdp/+spec/gdplaunchpad/+subscribe [13:20] czajkowski, this is a regression. It may be related to the javascript choice-picker, but I think the real issue is that the code that decides to render them thinks you have to be a driver [13:20] czajkowski, The code really did you to be a driver two years ago, that was fixed, but I think the links were never updated [13:20] hmm [13:21] czajkowski, this broke 12 months ago when the blueprints switched to the choice picker for everything [13:22] sinzui: nods well at this time of cycle it's probably going to be used a lot [13:24] czajkowski, do you see the assignee edit icon next to your name? [13:26] sinzui: I do now on https://blueprints.qastaging.launchpad.net/gdp/+spec/gdplaunchpad === jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jcsackett, rick_h* | Firefighting: - | Critical bugtasks: 4*10 [13:51] * jcsackett almost forgot it was thursday. [14:24] czajkowski, do you manage a project on qastaging that would could create a blueprint for and make me the assignee? [14:24] * sinzui wrote a test that shows the assignee gets the edit link [14:25] I dont manage any project [14:26] czajkowski, https://qastaging.launchpad.net/projects/+new register a project, configure blueprints, create one, make me the assignee [14:27] czajkowski, This widget we are looking at happen to be the only widget that does not work for IE...maybe the problem is bigger than we thought [14:27] ok [14:27] PS I care because we are using this widget to replace the privacy/security checkboxes [14:29] ok created now to configure blueprint [14:30] sinzui: dont see anything to configure bp [14:32] czajkowski, its on the right side of the project page, and the link is repeated on the blueprint page [14:34] sinzui: otp [14:34] ack [14:36] adeuring: You wanted me to review https://code.launchpad.net/~adeuring/lazr.jobrunner/use_job_repr_in_logging/+merge/98821 ? [14:37] abentley: right [14:38] abentley: and https://code.launchpad.net/~adeuring/launchpad/lp-lazr.jobrunner/+merge/97458 [14:38] adeuring: It looks a bit strange. Most reprs either resemble instance construction, like "BranchScanJob(id=5)" or have angle brackets like "". [14:38] sinzui: https://blueprints.qastaging.launchpad.net/conference-planning/+spec/test [14:39] abentley: well, we can use that too; I just wanted minimal changes that don't require related changes in the tests ;) [14:39] adeuring: Just "BranchScanJob" doesn't look like a repr to me. [14:39] czajkowski, :( it looks right. I see the definition and implementation edit icons [14:40] and I just changed one [14:40] * sinzui switches to firefox [14:40] abentley: I agree. But that's how it looked so far ;) (unless the derived class had its own __repr__ method) [14:41] bugger. Why does this work for me [14:43] adeuring: Is the job_str method really related to this? [14:45] abentley: yes, is it used in the logger calls [14:45] ...in runJob() and runJobHandleError() [14:46] adeuring: It doesn't seem to be used in the messages about MembershipNotificationJob that you changed, for example. [14:49] abentley: job_str() is used in runJob() and runJobHandlerError(), in self.logger.info(). [14:53]