[00:04] $ grep BugsInformationTypeMigrator garbo-hourly.log-20120320 | grep -c Blocked [00:04] 117 [00:04] wgrant: ^ [00:16] StevenK: Well, the problem here is clear. [00:17] The PRF has gone insane. Get it killed. === cjwatson_ is now known as cjwatson [01:23] wallyworld: So, using StormRangeFactory will generate a query something like https://pastebin.canonical.com/62588/ [01:24] wallyworld: Note the slight oddness in that you have to include the sort key in the SELECT list, due to the DISTINCT. [01:25] wgrant: ok, seems simple enough. how do i tell the batching infrastructure to use it? [01:26] wallyworld: There's only a couple of places that use it so far, but see eg. ArchiveSubscribersView [01:26] self.batchnav = BatchNavigator( [01:26] self.subscriptions, self.request, [01:26] range_factory=StormRangeFactory(self.subscriptions)) [01:26] ok, thanks [01:29] That is up to 30% faster than the original query. [01:30] Then you just need https://pastebin.canonical.com/62590/, which takes <1ms for 100 people. [02:15] Intriguing. [02:15] Chromium is of the opinion that my launchpad.dev wildcard cert has been revoked. [02:15] Despite it not having a CRL or OCSP endpoint specified. [03:44] Anyone feel like reviewing https://code.launchpad.net/~wgrant/launchpad/involvement-config-validity/+merge/98328? [03:47] wgrant: r=me [03:48] wgrant: In my branch to change IBug.{private,security_related} to make use of information_type most of the TestBugMirrorAccessTriggers fail with AttributeError: 'NoneType' object has no attribute 'concrete_artifact' [03:49] StevenK: You'll need to change the triggers to use information_type instead of private and security_related [03:49] StevenK: There's a card in Next for that. [03:49] I was going to do it once production's column was populated. [03:49] Right. [03:50] But is sort of seperate to this work to move IBug definition [03:51] It means that setting .private and .security_related still has to set the DB flags. [03:51] wgrant: I'm tempted to re-write the garbo jobs __call__ to be faster and not call bug._setInformationType() in a loop [03:52] See how it goes overnight. [03:52] Killing PRF before you sleep if necessary. [03:53] wgrant: What about the VACUUMs that keep turning up? [03:54] They shouldn't be too terrible most of the time. [03:58] StevenK: what about it ? [04:01] lifeless: They seem to keep blocking garbo jobs [04:01] Just look at the trail of fail in garbo-hourly.log [04:02] autovacuum is more a symptom not a cause [04:02] it canm be blocked by other writes [04:02] even if it shows up, its almost certainly not the reason for lag [04:03] Lag is not relevant here. [04:04] Long transactions. [04:04] samesame [04:04] vacuum is only long when its blocked on another xaction [04:54] StevenK: /win 6 [04:54] Bah [04:54] Haha [04:55] StevenK: How are you structuring your branches? [04:56] wgrant: I'm only working on one at the moment, and it's only 200 lines. [04:56] StevenK: Hm [04:56] wgrant: What are you thinking? [04:56] StevenK: But you appear to have a branch which stops writing to Bug.private and Bug.security_related [04:56] Which can't be in the same branch. [04:56] Not any more. [04:56] Ah [04:57] Good good. [04:57] IBug.private is now IBug._private and stuff is looking good. [04:59] Just running test -vvm bugs to see if there is anything heinous before I put up an MP. [04:59] This branch still keeps the flags and information_type in sync, just from the opposite direction? [05:00] Roughly [05:00] setPrivacyAndSecurityRelated is utterly horrid [05:00] So the less I touch it, the better. [05:01] I'm thinking we want to replace it with IBug.transitionInformationType() like the methods on IBugTask. [05:02] Yes. [05:03] 1) Get things keeping information_type in sync with private and security_related (done) [05:03] 2) Populate information_type (nearly done) [05:03] 3) Treat information_type as canonical, mirroring to private and security_related [05:03] 4) Get that deployed everywhere (including ftpmaster) [05:03] 5) Replace the triggers (I'm just testing that now) [05:04] 6) Stop touching private and security_related in any way [05:04] 7) Drop private and security_related [05:05] 8) Kill IBug.set{Private,SecurityRelated,PrivacyAndSecurityRelated}, replacing it with IBug.transistionToInformationType() [05:05] That's part of 6 [05:05] Ah [05:05] I did wonder [05:05] Right, I do like that plan. [05:06] We have cards for part 6 and 7? [05:07] Not yet. [05:07] Shall I create two in Backlog, sans bugs for now? [05:07] Sure. [05:08] Also, I missed a step [05:08] 2.5) ALTER TABLE Bug ALTER COLUMN information_type SET NOT NULL; [05:08] * StevenK stabs Kanban [05:08] Ah yes [05:08] Let me create a card for that and stuff into next [05:09] We'll need to fix the model/interface code seperately, but that's a trivial patch. [05:09] Are you fixing test_bug_mirror_access_triggers to not set private/security_related directly? [05:09] Already done [05:09] s/need/potentially and non-critically want/ [05:09] StevenK: Can you push? My trigger branch needs that. [05:09] wgrant: What I mean is sprinkle things like notNull=True to information_type in the model [05:10] StevenK: I'm aware. We probably want to do that at some point, but it's not in the critical path. [05:10] Nor important at all. [05:10] I'd prefer to do it just after the DB patch is deployed, but meh [05:11] The following 1 account(s) were found. Select the account you wish to access. [05:11] [05:11] * StevenK stabs Kanban harder [05:11] I only had one. [05:16] wgrant: Sorry, got distracted by Kanban. lp:~stevenk/launchpad/bugs-use-information_type [05:18] Thankyou sir. [05:25] StevenK: We should complete (and test as a whole) parts 3-7 before we land any of them. [05:25] In case we've missed something [05:25] 5 is done, asounds like you're close to 3 and 6, and 7 is pretty trivial :) [05:26] 3 yes, 6 no. [05:32] StevenK: Isn't 6 mostly a matter of deleting code? [05:32] Or does 3 not change the queries that use it? [05:33] 3 does not. I'm trying to keep it simple. [05:33] Ah, sure. [05:33] 3 falls back to _private and _security_related if information_type is not set, so it's not dependant on the garbo job being complete. [05:35] Hmmmm. That could not be helping. [06:02] wgrant: What should I set information_type to if IProduct.private_bugs -- I want to say PROPRIETARY. [06:03] StevenK: USERDATA [06:03] PROPRIETARY doesn't exist yet. [06:03] It's defined, but it doesn't exist. [06:56] wgrant: Are you still reviewing since you're OCR and everything. === almaisan-away is now known as al-maisan === al-maisan is now known as almaisan-away [07:51] LINE 3: ... WHERE bt.bug=bug.id ORDER BY bt.name LI... [07:51] Could have sworn we fixed that one already (BugTask.name does not exist...) [07:56] Scoping rules seem subtly different in 9.1 if you use the same tag to alias more than one table in a nested query [08:54] good morning [09:03] aloha [09:09] moin adeuring, czajkowski === jelmer_ is now known as jelmer === almaisan-away is now known as al-maisan [09:44] hi jtv, are you around? [09:45] Hi dpm! Trying to have a conference call atm, but can get back to you in half an hour or so if that's okay. [09:46] jtv, sure! if you've got 5 minutes for me later on, that'd be perfect === gmb changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: gmb | Firefighting: - | Critical bugtasks: 4*10 === jml` is now known as jml === mpt_ is now known as mpt [11:15] morning [11:44] Morning, everyone. [11:48] morning === al-maisan is now known as almaisan-away [13:32] abentley, adeuring, rick_h -- I'm coming. joining hangouts is slooooowww here for some reason. [13:33] deryck: all the beach air interfering with your bandwidth :P [13:51] abentley: can't hear you [13:51] adeuring: My mic shows levels. Let's both rejoin. [13:51] ok [13:53] abentley: let's use muble. Google claims that have to install an new plugin... === almaisan-away is now known as al-maisan [15:16] adeuring: I think job.fail needs transaction.abort before it. Here's how it fails without it: http://pastebin.ubuntu.com/892304/ [15:16] abentley: ouc, of course, yes [15:17] adeuring: Maybe we should go back to having commit/abort methods on the runner? I know it's not pretty. [15:19] abentley: I think we should first try to fix the bugs we have currnently. I really don't like to have DB commit into lazr.jobrunner... [15:19] adeuring: Or maybe the runner could accept a transaction manager. [15:19] as a parameter. [15:19] abentley: well, that's bascially teh same ;) [15:20] adeuring: This is a bug that we have currently. [15:20] adeuring: I was getting this, so I had to disable oops handling to get the real error. [15:21] abentley: does this mean that the OOPS handling is also broken? [15:22] adeuring: The oops handling was broken because job.fail was not aborting the transaction, so when it tried to invoke methods on job, it got InternalError: current transaction is aborted, commands ignored until end of transaction block [15:23] abentley: ah, ok. [15:24] adeuring: I've worked around all of this by doing "import transaction; transaction.abort()" immediately before job.fail, but this is not a long-term solution. [15:25] abentley: the branch I'm working on should fix the problem: it changes the methods fail(), queue(), suspend() etc [15:26] adeuring: Great. === al-maisan is now known as almaisan-away === matsubara is now known as matsubara-lunch [15:58] abentley, rick_h -- Just FYI for the YUI history curious, our phantom history events is a bug fixed in YUI 3.4.1. [15:59] deryck: awesome [15:59] deryck: Ah, great. [15:59] abentley, rick_h -- http://pastebin.ubuntu.com/892344/ shows the issue. simple initialState bug. [15:59] abentley, rick_h -- I vote for waiting on upgrade to fix it. :) [16:00] deryck: yea, guess so [16:01] only other option is to fork our 3.3.0 and patch the history module. [16:01] right, messy [16:01] then we have a diff against 3.3.0, have to maintain our own build, etc. when the issue is annoying but relatively minor. === salgado is now known as salgado-lunch === matsubara-lunch is now known as matsubara [16:57] deryck, rick_h what is stopping us from using 3.4.1? [16:57] sinzui, upgrade time && too many js irons in the fire. [16:58] sinzui: hopefully the combo loader will make it much easier to swap out versions, run all our tests, and find out what issues pop up [16:58] :( this is why we stopped updating zope for 3 years [16:59] rick_h, well if the calendar dies, we have incentive to use 3.4.1 with a hack to make the popup work in firefox [16:59] sinzui: that and 3.5 is due next month so the hope is more to jump to 3.5 [16:59] That would make me happy [16:59] I don't think it's far off actually. Just not NOW()! :) [17:00] sinzui: yea, so the plan isn't to not upgrade, but try tomake one big jump with combo loader + 3.5 in sucession [17:00] that's the chief driver of the combo loader, to make upgrades much easier. [17:01] yea, it's all kind of tied together with combo loader, 3.5, handlbars/pybars. Lots of parts moving that hopefully we can snap together [17:02] exactly. [17:02] and that's also what gives me hope that an upgrade is not just vapor, but something that will happen. === salgado-lunch is now known as salgado [17:34] tumbleweed: ping [18:42] jcsackett, do you have a few minutes to talk about the auditor editing case in the the mockups [18:59] sinzui: sure. [18:59] sinzui: can you start the hangout? i do not have a functioning webcam, so will have to use my phone. [19:01] jcsackett, starting now [19:03] jcsackett, does you phone see a hangout with me in the messenger? [19:13] rick_h: pong [19:14] tumbleweed: wanted to see if you could help me out with the test data so I can get the test working for that MP [19:14] I posted back on the MP what I got going, and need to see if the test data in wadllib is out of date [19:16] rick_h: I can't see pastebin.canonical.com [19:16] tumbleweed: k, sec [19:17] http://paste.mitechie.com/show/569/ [19:17] basically checking the date formats given the test json data in wadllib works out, so curious if the json out of that bug is different and needs updating [19:19] thanks for actually doing a test. I was just testing it from launchpadlib, against lp [19:20] flacoste: hey [19:20] tumbleweed: ok, I wasn't sure what the test is hitting and wondered if you could shoot me a json file to check against [19:20] hi lifeless! [19:20] yes, it would be a good time to talk! [19:20] tumbleweed: end of the day, the change seems good, but I'd like to get a good test to make sure we don't break things again [19:20] rick_h: IIRC the problem was that namespace_url (line 483 of application.py) was None [19:21] cool, @ 30 past? Would like to nab a snack first [19:22] * lifeless is astonished that pg 8.4 handles bugtag.name without blowing up [19:22] tumbleweed: ugh, ok, so deeper issue than it seemed [19:23] tumbleweed: ok, the source for that test stuff is open right? if so I guess I'll just pull that down and trace through to be able to duplicate something [19:24] rick_h: you should be able to test it by checking isinstance(lp.distributions['ubuntu'].date_created, datetime.datetime) [19:26] tumbleweed: right, but I need to get that into a unit tests with a json api response and an existing known wadl file. but yea, I'll pull that down and try to get it going [19:26] tumbleweed: thank [19:26] of course :) [19:26] it helps to know that's what the bug was hiting, couldn't tell that input the test file in the pastebin from the bug was using as input [19:32] flacoste: okies [19:33] * flacoste starts hang out [19:35] gary_poster: do you have a little time today, once I'm finished with flacoste, to talk paralleltest ? [19:36] lifeless, I have my call with Francis in 24 min, and am trying to get a machine ready for analysis right now. I know it would be good to talk. We also have our scheduled call tomorrow though [19:42] IOW, not easily, now [19:42] no [20:06] gary_poster: no worries; perhaps we can talk a little tomorrow 1:1 [20:06] ok cool [20:06] lifeless I'll ping if I have time before EoD today in case you are around/available [20:10] cool, thanks [20:55] lifeless, if we can make it fast we could talk now [20:55] lifeless, https://talkgadget.google.com/hangouts/extras/canonical.com/goldenhordeoneonone ? [20:57] getting some weird LP oopses ... anyone around to take a look? [20:58] kirkland: gotta link the OOPS man :P [20:58] gary_poster: sure, trying to get in [20:58] this multi account stuff really is a nuisannce [20:58] lifeless: copying over from #launchpad (where I did) ... [20:58] Sorry, something went wrong with your search. (Error ID: OOPS-3611440967e645c2a3d8c68345be9b24) [20:59] lifeless, I tried inviting your @gmail.com [20:59] yeah [20:59] I'm also getting a "not found" on a bug that I've been looking at/working on all day [20:59] that used to work [20:59] seems to not now [20:59] gary_poster: sending you an invite instead [21:00] :-) cool [21:00] kirkland: thats a 404; it is oopsing because you have a local referrer [21:14] lifeless: what does that mean, sorry for being dense? [21:14] lifeless: I've been working on this private bug all day long https://bugs.launchpad.net/ztrustee/+bug/959650 [21:14] lifeless: and it's now disappeared for all I can tell [21:18] i guess you no longer have permission to see it [21:18] for whatever reason [21:28] kirkland: you may have been unsubscribed from it [21:29] kirkland: hmm [21:29] ztrustee may have been renamed :) [21:29] https://launchpad.net/ztrustree [21:30] lifeless: interesting; I'm on the team that owns the project [21:30] kirkland: can you see the project? [21:30] lifeless: I can see ztrustee (which is right); ztrustree is 404 [21:31] hah, yes ok [21:31] so there are two possibilities [21:31] Include the error ID OOPS-e72a16ca0af50bf4338dddc14d2ba243 in your message. [21:31] we've nerfed something in private projects [21:31] lifeless: any info there? [21:31] or you / your team have been unsubscribed from that bug, leading to no access [21:32] lifeless: how odd, okay, thanks [21:32] lifeless: we can recreate the bug, that's no problem [21:32] its failing in File "/srv/launchpad.net/production/launchpad-rev-14969/lib/lp/bugs/security.py", line 127, in checkAuthenticated [21:32] return self.obj.userCanView(user.person) [21:32] which is a cannot-access situation [21:33] lifeless: this is the first time something like this has happened, though, so I thought I'd mention it in case it's systemic [21:33] kirkland: I'm 99% certain someone in your team unsubscribed the team from the bug [21:33] lifeless: okay, we'll go with that [21:33] lifeless: I'll re-raise if it happens unexpectedly again [21:33] what they need to do if they don't want notifications is to use the *mute* feature [21:33] until the disclosure project is rolled out [21:33] the disclosure project is decoupling access controll and notification [21:33] lifeless: k [21:33] which will avoid this rather spectacular foot gun [21:34] lifeless: btw, can you tell me who to thank for the recent "Affiliations" feature, when searching for a person or team? [21:34] lifeless: cool [21:34] purple panthers [21:34] the same folk working on disclosure [21:34] e.g. sinzui & co [21:37] lifeless: lovely [21:38] sinzui: thanks!!! [21:42] kirkland, I will pass that on to the squad. they did the testing and work to make it appear in real time [21:42] sinzui: it's *awesome* [21:43] sinzui: and exactly what I was looking for in Bug #930364, which was marked wont-fix [21:43] <_mup_> Bug #930364: show badges when searching/selecting a launchpad user

< https://launchpad.net/bugs/930364 > [21:44] sinzui: I think you noted that in Comment 7, which is good enough for me! [21:44] fab [21:47] kirkland, we have started the UI for showing who has access to private data: https://qastaging.launchpad.net/ecryptfs/+sharing is a draft of a what we will announce in a few weeks [21:47] * sinzui cannot see that page so cannot answer question about who has access and how, but the views to do that will be in place next week [22:30] StevenK, http://blog.launchpad.net/general/reimagining-the-nature-of-privacy-in-launchpad-part-1