[00:04] <StevenK> $ grep BugsInformationTypeMigrator garbo-hourly.log-20120320 | grep -c Blocked
[00:04] <StevenK> 117
[00:04] <StevenK> wgrant: ^
[00:16] <wgrant> StevenK: Well, the problem here is clear.
[00:17] <wgrant> The PRF has gone insane. Get it killed.
[01:23] <wgrant> wallyworld: So, using StormRangeFactory will generate a query something like https://pastebin.canonical.com/62588/
[01:24] <wgrant> wallyworld: Note the slight oddness in that you have to include the sort key in the SELECT list, due to the DISTINCT.
[01:25] <wallyworld> wgrant: ok, seems simple enough. how do i tell the batching infrastructure to use it?
[01:26] <wgrant> wallyworld: There's only a couple of places that use it so far, but see eg. ArchiveSubscribersView
[01:26] <wgrant>         self.batchnav = BatchNavigator(
[01:26] <wgrant>             self.subscriptions, self.request,
[01:26] <wgrant>             range_factory=StormRangeFactory(self.subscriptions))
[01:26] <wallyworld> ok, thanks
[01:29] <wgrant> That is up to 30% faster than the original query.
[01:30] <wgrant> Then you just need https://pastebin.canonical.com/62590/, which takes <1ms for 100 people.
[02:15] <wgrant> Intriguing.
[02:15] <wgrant> Chromium is of the opinion that my launchpad.dev wildcard cert has been revoked.
[02:15] <wgrant> Despite it not having a CRL or OCSP endpoint specified.
[03:44] <wgrant> Anyone feel like reviewing https://code.launchpad.net/~wgrant/launchpad/involvement-config-validity/+merge/98328?
[03:47] <StevenK> wgrant: r=me
[03:48] <StevenK> wgrant: In my branch to change IBug.{private,security_related} to make use of information_type most of the TestBugMirrorAccessTriggers fail with AttributeError: 'NoneType' object has no attribute 'concrete_artifact'
[03:49] <wgrant> StevenK: You'll need to change the triggers to use information_type instead of private and security_related
[03:49] <wgrant> StevenK: There's a card in Next for that.
[03:49] <wgrant> I was going to do it once production's column was populated.
[03:49] <StevenK> Right.
[03:50] <StevenK> But is sort of seperate to this work to move IBug definition
[03:51] <wgrant> It means that setting .private and .security_related still has to set the DB flags.
[03:51] <StevenK> wgrant: I'm tempted to re-write the garbo jobs __call__ to be faster and not call bug._setInformationType() in a loop
[03:52] <wgrant> See how it goes overnight.
[03:52] <wgrant> Killing PRF before you sleep if necessary.
[03:53] <StevenK> wgrant: What about the VACUUMs that keep turning up?
[03:54] <wgrant> They shouldn't be too terrible most of the time.
[03:58] <lifeless> StevenK: what about it ?
[04:01] <StevenK> lifeless: They seem to keep blocking garbo jobs
[04:01] <StevenK> Just look at the trail of fail in garbo-hourly.log
[04:02] <lifeless> autovacuum is more a symptom not a cause
[04:02] <lifeless> it canm be blocked by other writes
[04:02] <lifeless> even if it shows up, its almost certainly not the reason for lag
[04:03] <wgrant> Lag is not relevant here.
[04:04] <wgrant> Long transactions.
[04:04] <lifeless> samesame
[04:04] <lifeless> vacuum is only long when its blocked on another xaction
[04:54] <wgrant> StevenK: /win 6
[04:54] <wgrant> Bah
[04:54] <StevenK> Haha
[04:55] <wgrant> StevenK: How are you structuring your branches?
[04:56] <StevenK> wgrant: I'm only working on one at the moment, and it's only 200 lines.
[04:56] <wgrant> StevenK: Hm
[04:56] <StevenK> wgrant: What are you thinking?
[04:56] <wgrant> StevenK: But you appear to have a branch which stops writing to Bug.private and Bug.security_related
[04:56] <wgrant> Which can't be in the same branch.
[04:56] <StevenK> Not any more.
[04:56] <wgrant> Ah
[04:57] <wgrant> Good good.
[04:57] <StevenK> IBug.private is now IBug._private and stuff is looking good.
[04:59] <StevenK> Just running test -vvm bugs to see if there is anything heinous before I put up an MP.
[04:59] <wgrant> This branch still keeps the flags and information_type in sync, just from the opposite direction?
[05:00] <StevenK> Roughly
[05:00] <StevenK> setPrivacyAndSecurityRelated is utterly horrid
[05:00] <StevenK> So the less I touch it, the better.
[05:01] <StevenK> I'm thinking we want to replace it with IBug.transitionInformationType() like the methods on IBugTask.
[05:02] <wgrant> Yes.
[05:03] <wgrant> 1) Get things keeping information_type in sync with private and security_related (done)
[05:03] <wgrant> 2) Populate information_type (nearly done)
[05:03] <wgrant> 3) Treat information_type as canonical, mirroring to private and security_related
[05:03] <wgrant> 4) Get that deployed everywhere (including ftpmaster)
[05:03] <wgrant> 5) Replace the triggers (I'm just testing that now)
[05:04] <wgrant> 6) Stop touching private and security_related in any way
[05:04] <wgrant> 7) Drop private and security_related
[05:05] <StevenK> 8) Kill IBug.set{Private,SecurityRelated,PrivacyAndSecurityRelated}, replacing it with IBug.transistionToInformationType()
[05:05] <wgrant> That's part of 6
[05:05] <StevenK> Ah
[05:05] <StevenK> I did wonder
[05:05] <StevenK> Right, I do like that plan.
[05:06] <StevenK> We have cards for part 6 and 7?
[05:07] <wgrant> Not yet.
[05:07] <StevenK> Shall I create two in Backlog, sans bugs for now?
[05:07] <wgrant> Sure.
[05:08] <wgrant> Also, I missed a step
[05:08] <wgrant> 2.5) ALTER TABLE Bug ALTER COLUMN information_type SET NOT NULL;
[05:08]  * StevenK stabs Kanban
[05:08] <StevenK> Ah yes
[05:08] <StevenK> Let me create a card for that and stuff into next
[05:09] <StevenK> We'll need to fix the model/interface code seperately, but that's a trivial patch.
[05:09] <wgrant> Are you fixing test_bug_mirror_access_triggers to not set private/security_related directly?
[05:09] <StevenK> Already done
[05:09] <wgrant> s/need/potentially and non-critically want/
[05:09] <wgrant> StevenK: Can you push? My trigger branch needs that.
[05:09] <StevenK> wgrant: What I mean is sprinkle things like notNull=True to information_type in the model
[05:10] <wgrant> StevenK: I'm aware. We probably want to do that at some point, but it's not in the critical path.
[05:10] <wgrant> Nor important at all.
[05:10] <StevenK> I'd prefer to do it just after the DB patch is deployed, but meh
[05:11] <StevenK> The following 1 account(s) were found. Select the account you wish to access.
[05:11] <StevenK> <list of two things>
[05:11]  * StevenK stabs Kanban harder
[05:11] <wgrant> I only had one.
[05:16] <StevenK> wgrant: Sorry, got distracted by Kanban. lp:~stevenk/launchpad/bugs-use-information_type
[05:18] <wgrant> Thankyou sir.
[05:25] <wgrant> StevenK: We should complete (and test as a whole) parts 3-7 before we land any of them.
[05:25] <wgrant> In case we've missed something
[05:25] <wgrant> 5 is done, asounds like you're close to 3 and 6, and 7 is pretty trivial :)
[05:26] <StevenK> 3 yes, 6 no.
[05:32] <wgrant> StevenK: Isn't 6 mostly a matter of deleting code?
[05:32] <wgrant> Or does 3 not change the queries that use it?
[05:33] <StevenK> 3 does not. I'm trying to keep it simple.
[05:33] <wgrant> Ah, sure.
[05:33] <StevenK> 3 falls back to _private and _security_related if information_type is not set, so it's not dependant on the garbo job being complete.
[05:35] <StevenK> Hmmmm. That could not be helping.
[06:02] <StevenK> wgrant: What should I set information_type to if IProduct.private_bugs -- I want to say PROPRIETARY.
[06:03] <wgrant> StevenK: USERDATA
[06:03] <wgrant> PROPRIETARY doesn't exist yet.
[06:03] <wgrant> It's defined, but it doesn't exist.
[06:56] <StevenK> wgrant: Are you still reviewing since you're OCR and everything.
[07:51] <stub>  LINE 3: ...                     WHERE bt.bug=bug.id ORDER BY bt.name LI...
[07:51] <stub> Could have sworn we fixed that one already (BugTask.name does not exist...)
[07:56] <stub> Scoping rules seem subtly different in 9.1 if you use the same tag to alias more than one table in a nested query
[08:54] <adeuring> good morning
[09:03] <czajkowski> aloha
[09:09] <jelmer_> moin adeuring, czajkowski
[09:44] <dpm> hi jtv, are you around?
[09:45] <jtv> Hi dpm!  Trying to have a conference call atm, but can get back to you in half an hour or so if that's okay.
[09:46] <dpm> jtv, sure! if you've got 5 minutes for me later on, that'd be perfect
[11:15] <rick_h> morning
[11:44] <deryck> Morning, everyone.
[11:48] <rick_h> morning
[13:32] <deryck> abentley, adeuring, rick_h -- I'm coming.  joining hangouts is slooooowww here for some reason.
[13:33] <rick_h> deryck: all the beach air interfering with your bandwidth :P
[13:51] <adeuring> abentley: can't hear you
[13:51] <abentley> adeuring: My mic shows levels.  Let's both rejoin.
[13:51] <adeuring> ok
[13:53] <adeuring> abentley: let's use muble. Google claims that have to install an new plugin...
[15:16] <abentley> adeuring: I think job.fail needs transaction.abort before it.  Here's how it fails without it: http://pastebin.ubuntu.com/892304/
[15:16] <adeuring> abentley: ouc, of course, yes
[15:17] <abentley> adeuring: Maybe we should go back to having commit/abort methods on the runner?  I know it's not pretty.
[15:19] <adeuring> abentley: I think we should first try to fix the bugs we have currnently. I really don't like to have DB commit into lazr.jobrunner...
[15:19] <abentley> adeuring: Or maybe the runner could accept a transaction manager.
[15:19] <abentley> as a parameter.
[15:19] <adeuring> abentley: well, that's bascially teh same ;)
[15:20] <abentley> adeuring: This is a bug that we have currently.
[15:20] <abentley> adeuring: I was getting this, so I had to disable oops handling to get the real error.
[15:21] <adeuring> abentley: does this mean that the OOPS handling is also broken?
[15:22] <abentley> adeuring: The oops handling was broken because job.fail was not aborting the transaction, so when it tried to invoke methods on job, it got InternalError: current transaction is aborted, commands ignored until end of transaction block
[15:23] <adeuring> abentley: ah, ok.
[15:24] <abentley> adeuring: I've worked around all of this by doing "import transaction; transaction.abort()" immediately before job.fail, but this is not a long-term solution.
[15:25] <adeuring> abentley: the branch I'm working on should fix the problem: it changes the methods fail(), queue(), suspend() etc
[15:26] <abentley> adeuring: Great.
[15:58] <deryck> abentley, rick_h -- Just FYI for the YUI history curious, our phantom history events is a bug fixed in YUI 3.4.1.
[15:59] <rick_h> deryck: awesome
[15:59] <abentley> deryck: Ah, great.
[15:59] <deryck> abentley, rick_h -- http://pastebin.ubuntu.com/892344/ shows the issue.  simple initialState bug.
[15:59] <deryck> abentley, rick_h -- I vote for waiting on upgrade to fix it. :)
[16:00] <rick_h> deryck: yea, guess so
[16:01] <deryck> only other option is to fork our 3.3.0 and patch the history module.
[16:01] <rick_h> right, messy
[16:01] <deryck> then we have a diff against 3.3.0, have to maintain our own build, etc. when the issue is annoying but relatively minor.
[16:57] <sinzui> deryck, rick_h what is stopping us from using 3.4.1?
[16:57] <deryck> sinzui, upgrade time && too many js irons in the fire.
[16:58] <rick_h> sinzui: hopefully the combo loader will make it much easier to swap out versions, run all our tests, and find out what issues pop up
[16:58] <sinzui> :( this is why we stopped updating zope for 3 years
[16:59] <sinzui> rick_h, well if the calendar dies, we have incentive to use 3.4.1 with a hack to make the popup work in firefox
[16:59] <rick_h> sinzui: that and 3.5 is due next month so the hope is more to jump to 3.5
[16:59] <sinzui> That would make me happy
[16:59] <deryck> I don't think it's far off actually.  Just not NOW()! :)
[17:00] <rick_h> sinzui: yea, so the plan isn't to not upgrade, but try tomake one big jump with combo loader + 3.5 in sucession
[17:00] <deryck> that's the chief driver of the combo loader, to make upgrades much easier.
[17:01] <rick_h> yea, it's all kind of tied together with combo loader, 3.5, handlbars/pybars. Lots of parts moving that hopefully we can snap together
[17:02] <deryck> exactly.
[17:02] <deryck> and that's also what gives me hope that an upgrade is not just vapor, but something that will happen.
[17:34] <rick_h> tumbleweed: ping
[18:42] <sinzui> jcsackett, do you have a few minutes to talk about the auditor editing case in the the mockups
[18:59] <jcsackett> sinzui: sure.
[18:59] <jcsackett> sinzui: can you start the hangout? i do not have a functioning webcam, so will have to use my phone.
[19:01] <sinzui> jcsackett, starting now
[19:03] <sinzui> jcsackett, does you phone see a hangout with me in the messenger?
[19:13] <tumbleweed> rick_h: pong
[19:14] <rick_h> tumbleweed: wanted to see if you could help me out with the test data so I can get the test working for that MP
[19:14] <rick_h> I posted back on the MP what I got going, and need to see if the test data in wadllib is out of date
[19:16] <tumbleweed> rick_h: I can't see pastebin.canonical.com
[19:16] <rick_h> tumbleweed: k, sec
[19:17] <rick_h> http://paste.mitechie.com/show/569/
[19:17] <rick_h> basically checking the date formats given the test json data in wadllib works out, so curious if the json out of that bug is different and needs updating
[19:19] <tumbleweed> thanks for actually doing a test. I was just testing it from launchpadlib, against lp
[19:20] <lifeless> flacoste: hey
[19:20] <rick_h> tumbleweed: ok, I wasn't sure what the test is hitting and wondered if you could shoot me a json file to check against
[19:20] <flacoste> hi lifeless!
[19:20] <flacoste> yes, it would be a good time to talk!
[19:20] <rick_h> tumbleweed: end of the day, the change seems good, but I'd like to get a good test to make sure we don't break things again
[19:20] <tumbleweed> rick_h: IIRC the problem was that namespace_url (line 483 of application.py) was None
[19:21] <lifeless> cool, @ 30 past? Would like to nab a snack first
[19:22]  * lifeless is astonished that pg 8.4 handles bugtag.name without blowing up
[19:22] <rick_h> tumbleweed: ugh, ok, so deeper issue than it seemed
[19:23] <rick_h> tumbleweed: ok, the source for that test stuff is open right? if so I guess I'll just pull that down and trace through to be able to duplicate something
[19:24] <tumbleweed> rick_h: you should be able to test it by checking isinstance(lp.distributions['ubuntu'].date_created, datetime.datetime)
[19:26] <rick_h> tumbleweed: right, but I need to get that into a unit tests with a json api response and an existing known wadl file. but yea, I'll pull that down and try to get it going
[19:26] <rick_h> tumbleweed: thank
[19:26] <tumbleweed> of course :)
[19:26] <rick_h> it helps to know that's what the bug was hiting, couldn't tell that input the test file in the pastebin from the bug was using as input
[19:32] <lifeless> flacoste: okies
[19:33]  * flacoste starts hang out
[19:35] <lifeless> gary_poster: do you have a little time today, once I'm finished with flacoste, to talk paralleltest ?
[19:36] <gary_poster> lifeless, I have my call with Francis in 24 min, and am trying to get a machine ready for analysis right now.  I know it would be good to talk.  We also have our scheduled call tomorrow though
[19:42] <gary_poster> IOW, not easily, now
[19:42] <gary_poster> no
[20:06] <lifeless> gary_poster: no worries; perhaps we can talk a little tomorrow 1:1
[20:06] <gary_poster> ok cool
[20:06] <gary_poster> lifeless I'll ping if I have time before EoD today in case you are around/available
[20:10] <lifeless> cool, thanks
[20:55] <gary_poster> lifeless, if we can make it fast we could talk now
[20:55] <gary_poster> lifeless, https://talkgadget.google.com/hangouts/extras/canonical.com/goldenhordeoneonone ?
[20:57] <kirkland> getting some weird LP oopses ... anyone around to take a look?
[20:58] <lifeless> kirkland: gotta link the OOPS man :P
[20:58] <lifeless> gary_poster: sure, trying to get in
[20:58] <lifeless> this multi account stuff really is a nuisannce
[20:58] <kirkland> lifeless: copying over from #launchpad (where I did) ...
[20:58] <kirkland> Sorry, something went wrong with your search. (Error ID: OOPS-3611440967e645c2a3d8c68345be9b24)
[20:59] <gary_poster> lifeless, I tried inviting your @gmail.com
[20:59] <lifeless> yeah
[20:59] <kirkland> I'm also getting a "not found" on a bug that I've been looking at/working on all day
[20:59] <lifeless> that used to work
[20:59] <lifeless> seems to not now
[20:59] <lifeless> gary_poster: sending you an invite instead
[21:00] <gary_poster> :-) cool
[21:00] <lifeless> kirkland: thats a 404; it is oopsing because you have a local referrer
[21:14] <kirkland> lifeless: what does that mean, sorry for being dense?
[21:14] <kirkland> lifeless: I've been working on this private bug all day long https://bugs.launchpad.net/ztrustee/+bug/959650
[21:14] <kirkland> lifeless: and it's now disappeared for all I can tell
[21:18] <mwhudson> i guess you no longer have permission to see it
[21:18] <mwhudson> for whatever reason
[21:28] <lifeless> kirkland: you may have been unsubscribed from it
[21:29] <lifeless> kirkland: hmm
[21:29] <lifeless> ztrustee may have been renamed :)
[21:29] <lifeless> https://launchpad.net/ztrustree
[21:30] <kirkland> lifeless: interesting;  I'm on the team that owns the project
[21:30] <lifeless> kirkland: can you see the project?
[21:30] <kirkland> lifeless: I can see ztrustee (which is right);  ztrustree is 404
[21:31] <lifeless> hah, yes ok
[21:31] <lifeless> so there are two possibilities
[21:31] <kirkland> Include the error ID OOPS-e72a16ca0af50bf4338dddc14d2ba243 in your message.
[21:31] <lifeless> we've nerfed something in private projects
[21:31] <kirkland> lifeless: any info there?
[21:31] <lifeless> or you / your team have been unsubscribed from that bug, leading to no access
[21:32] <kirkland> lifeless: how odd, okay, thanks
[21:32] <kirkland> lifeless: we can recreate the bug, that's no problem
[21:32] <lifeless> its failing in   File "/srv/launchpad.net/production/launchpad-rev-14969/lib/lp/bugs/security.py", line 127, in checkAuthenticated
[21:32] <lifeless>     return self.obj.userCanView(user.person)
[21:32] <lifeless> which is a cannot-access situation
[21:33] <kirkland> lifeless: this is the first time something like this has happened, though, so I thought I'd mention it in case it's systemic
[21:33] <lifeless> kirkland: I'm 99% certain someone in your team unsubscribed the team from the bug
[21:33] <kirkland> lifeless: okay, we'll go with that
[21:33] <kirkland> lifeless: I'll re-raise if it happens unexpectedly again
[21:33] <lifeless> what they need to do if they don't want notifications is to use the *mute* feature
[21:33] <lifeless> until the disclosure project is rolled out
[21:33] <lifeless> the disclosure project is decoupling access controll and notification
[21:33] <kirkland> lifeless: k
[21:33] <lifeless> which will avoid this rather spectacular foot gun
[21:34] <kirkland> lifeless: btw, can you tell me who to thank for the recent "Affiliations" feature, when searching for a person or team?
[21:34] <kirkland> lifeless: cool
[21:34] <lifeless> purple panthers
[21:34] <lifeless> the same folk working on disclosure
[21:34] <lifeless> e.g. sinzui & co
[21:37] <kirkland> lifeless: lovely
[21:38] <kirkland> sinzui: thanks!!!
[21:42] <sinzui> kirkland, I will pass that on to the squad. they did the testing and work to make it appear in real time
[21:42] <kirkland> sinzui: it's *awesome*
[21:43] <kirkland> sinzui: and exactly what I was looking for in Bug #930364, which was marked wont-fix
[21:43] <_mup_> Bug #930364: show badges when searching/selecting a launchpad user <disclosure> <person-picker> <Launchpad itself:Won't Fix> < https://launchpad.net/bugs/930364 >
[21:44] <kirkland> sinzui: I think you noted that in Comment 7, which is good enough for me!
[21:44] <sinzui> fab
[21:47] <sinzui> kirkland, we have started the UI for showing who has access to private data: https://qastaging.launchpad.net/ecryptfs/+sharing is a draft of a what we will announce in a few weeks
[21:47]  * sinzui cannot see that page so cannot answer question about who has access and how, but the views to do that will be in place next week
[22:30] <sinzui> StevenK, http://blog.launchpad.net/general/reimagining-the-nature-of-privacy-in-launchpad-part-1