[01:34] <lifeless> http://arstechnica.com/security/2012/06/flame-crypto-breakthrough/
[01:34] <lifeless> cjwatson: ^
[01:34] <mwhudson> i.e. it was NSA wot did it?
[01:35] <lifeless> something like
[01:35] <mwhudson> i like the first comment:
[01:35] <mwhudson> It is going to be both funny and sad if it turns out this thing was written by some 15 year old in his parent's basement.
[02:38] <cjwatson> I wonder how well the maths behind that will transfer to the SHA family :-/
[02:40] <wgrant> cjwatson: SHA-1 perhaps, but isn't SHA-2 fairly different?
[02:44] <wgrant> But I guess if it's an entirely novel form of the attack it may have some relevance.
[02:44] <cjwatson> They're similar enough for the multicollisions approach to still apply.
[02:45] <cjwatson> All the current hash families aside maybe from some elliptic curve ones have roughly the same general shape.
[02:46] <cjwatson> Which isn't to say that it wouldn't be quite a bit more world-class maths to transfer it ...
[02:46] <wgrant> Indeed.
[02:48] <cjwatson> My general takeaway from hash developments over the last ~10 years is that every deployed system needs a structure for moving to new hashes, anyway.
[02:48] <cjwatson> So meh :-)
[02:49] <cjwatson> Pretty surprised Microsoft is still vulnerable to MD5-based attacks, though.
[02:49] <cody-somerville> I'm curious. Might be a stupid question but why would the flame malware have this new world class crypto breakthrough in it anyhow? AFAIU, even though it's been discovered how to crack md5 it still takes a little while plus why would you need the crypto breakthrough in the malware yourself? Wouldn't you just use the math to create the fake cert then you use that to sign stuff?
[02:50] <mwhudson> i was wondering that
[02:51] <mwhudson> i think that the deduction that a new approach was used by looking at the way the form the colliding data has
[02:51] <mwhudson> reading between the lines of http://www.cwi.nl/news/2012/cwi-cryptanalist-discovers-new-cryptographic-attack-variant-in-flame-spy-malware
[02:51] <mwhudson> so there's not a novel md5 collision generating algorithm in flame
[02:52] <cjwatson> Ah, it's only Terminal Server that's still MD5y
[02:52] <wgrant> The collision code isn't in flame.
[02:53] <wgrant> They were just able to determine it was novel from the cert it uses.
[02:53] <wgrant> The known attacks are fairly distinctive :)
[04:10] <wgrant> wallyworld: Thanks for cleaning that up.
[04:11] <wallyworld> had to be done
[04:46] <cjwatson> wgrant: Have you had a chance to look at my queue-api branch?
[05:02] <wgrant> cjwatson: A little. Have you considered the performance impact of the additional exported properties on PackageUpload?
[05:02] <wgrant> Unless they're preloaded they will likely make it impossible to call getPackageUploads
[05:17] <lifeless> cjwatson: I want to talk to you about deb deltas
[05:18] <lifeless> cjwatson: are you up for a while, or is your circadian rhythm not ?
[05:22] <cjwatson> wgrant: I haven't.  Advice welcome
[05:22] <cjwatson> lifeless: Meh - I'm kind of here but mostly in a "hacking until my brain lets me go back to sleep" kind of way
[05:22] <cjwatson> So maybe now isn't the ideal time for deep thought
[05:23] <lifeless> cjwatson: ok, ping me when we cross paths and you're up for deep though
[05:23] <lifeless> t
[05:23] <cjwatson> lifeless: OK - can you send me an e-mail reminder?
[05:23] <lifeless> yes, when I'm not quite so crook
[05:23] <cjwatson> (Then at least it can get lost in the depths of my inbox as well as the depths of my brain ...)
[05:23]  * cjwatson nods
[05:25] <StevenK> lifeless: You have the entire weekend to be sick now :-P
[05:27] <StevenK> lifeless: Do you know anything about stub's GIN work?
[05:27] <cjwatson> wgrant: Is DecoratedResultSet(query, pre_iter_hook=...) the mechanism for this?
[05:28] <cjwatson> I find storm a little difficult to navigate unassisted.
[05:28] <StevenK> Welcome to the club
[05:28] <bigjools> it's not the most pleasant code
[05:28] <StevenK> Everytime I get a traceback in the depths of Storm, I tend to yell for help
[05:28] <wgrant> cjwatson: Yeah.
[05:29] <wgrant> Note that DRS is one of our layers on top of Storm.
[05:29] <wgrant> Storm rejected it.
[05:29] <wgrant> StevenK: What about GIN?
[05:30] <StevenK> wgrant: DB r11636 is up next -- [r=stub][bug=306201][incr] Switch BugTaskFlat.fti index from GiST to GIN
[05:31] <StevenK> But then DB r11640 says --- [r=stub][bug=1007333] pgstattuple doesn't support GIN indexes, so don't do that
[05:31] <wgrant> StevenK: Bah, I typoed the deployment request
[05:31] <wgrant> Should be 11639
[05:31]  * wgrant fixes.
[05:31] <StevenK> wgrant: Haha, what did you have?
[05:31] <wgrant> 11640
[05:32] <StevenK> WCPGW
[05:32] <wgrant> Nothing at all, since I believe 11640's was probably applied live.
[05:32] <wgrant> But I'm not sure.
[05:32] <wgrant> Part of 11636 was as well, but not all.
[05:33] <StevenK> We can probably check that and make db-stable's deployment report a bit happier?
[05:33]  * StevenK stabs wallyworld.
[05:34] <StevenK> Just when I want him, he buggers off.
[05:48] <StevenK> wallyworld_, wgrant: Test fixes for branch-subscribe-aag and checking that the reviewer isn't an open team for private branches: http://pastebin.ubuntu.com/1029885/
[05:49]  * wallyworld_ looks
[05:54] <wallyworld_> StevenK: test_open_reviewer_private_branch, i'd like to see test_closed_team_reviewer_private_branch also even thoug the functionality is implicitly tested elsewhere, it would be good to have the two matching tests for this together
[05:54] <wallyworld_> StevenK: also, add a comment to _acceptable_to_give_visibility
[05:54] <lifeless> wgrant: I don't know that DRS was ever offered to storm
[05:55] <wgrant> http://comments.gmane.org/gmane.comp.python.storm/1319
[05:57] <lifeless> oh right
[05:57] <lifeless> so yes, I remember, and disagree with them :)
[05:59] <StevenK> wallyworld_: http://pastebin.ubuntu.com/1029896/ should address both comments.
[06:00] <wallyworld_> StevenK: thanks, looks good
[06:18] <wallyworld_> StevenK: i have to go to soccer - i can +1 your mp when i get back
[06:37] <cjwatson> wgrant: Well, http://paste.ubuntu.com/1029949/ demonstrates the problem, but adding more load_referencing/load_related doesn't seem to reduce the query count that the test complains about, so I must be doing something wrong ...
[06:38] <StevenK> wallyworld_: It already was, I've tossed it to ec2
[06:40] <cjwatson> (The test output is http://paste.ubuntu.com/1029951/, which does look like a moderately plausible set of queries that might be happening here)
[06:40] <wgrant> cjwatson: Those load_relateds should be sufficient (Reference columns look up by primary key, so if an object is already in Storms cache it won't cause a DB query), but the load_referencing things aren't, since the problematic properties issue queries directly or use a ReferenceSet or SQLMultipleJoin, none of which can be cached in Storm currently.
[06:40] <wgrant> So you need to use @cachedproperty
[06:41] <wgrant> And populate the property cache explicitly with the result of load_referencing
[06:41] <wgrant> And then sob.
[06:41] <wgrant> cjwatson: eg. search for load_referencing in lib/lp/registry/vocabularies.py
[06:42] <cjwatson> ... I think I need sleep, then caffeine, then food, then beer, *then* to attack this
[06:42] <cjwatson> But thanks, that's exactly the kind of reference I was looking for
[06:43] <wgrant> Heh
[06:50] <wgrant> cjwatson: PersonSet._getPrecachedPersons may also be of interest.
[06:51] <wgrant> cjwatson: It's one of the earlier cases of this, so it's a bit terrible but roughly a sort of correct idea.
[07:04] <jam> Hey all, I'm getting a timeout trying to submit a merge proposal: The following errors were encountered: Timeout error, please try again in a few minutes.
[07:04] <jam> is there a downtime I'm unaware of?
[07:04] <wgrant> jam: No, that's a bug.
[07:05] <wgrant> What's the OOPS ID?
[07:05] <jam> wgrant: no oops
[07:05] <wgrant> (perhaps check the AJAX log in the top right corner.
[07:05] <jam> just failure
[07:05] <wgrant> There is an OOPS
[07:05] <jam> and it just succeeded :(
[07:05] <wgrant> It might just not be shown.
[07:05] <jam> I reproduced it 3 times, but yeah, no log of the oops for me to go check.
[07:05] <jam> I'll try back, but I think that will wipe ajax stuff
[07:06] <jam> yeah
[07:50] <adeuring> good morning
[07:52] <jtv> Hi adeuring
[07:52] <adeuring> hi jtv
[09:00] <stub> Test fix mode due to  lp.services.job.tests.test_celeryjob.TestRunMissingJobs.test_find_missing_ready on db-devel
[09:02] <stub> Pretty certain it is spurious - no sampledata landings in the timeframe and test passed on devel
[09:45]  * jml feels dumb for not running tests on db-rename-archive-commercial-to-suppress last night
[09:45] <jml> hmm. last build failed. looks like a network error.
[09:46] <jml> https://lpbuildbot.canonical.com/builders/lucid_db_lp/builds/2028/steps/shell_5/logs/stdio
[09:56] <gmb> adeuring, Do you have time to review https://code.launchpad.net/~gmb/launchpad/bug-1009712/+merge/109315 ?
[09:56] <adeuring> gmb: sure
[09:56] <gmb> Thanks
[10:02] <adeuring> gmb: the feature retlaed changes look good. but what is test_noise supposed to do?
[10:03] <gmb> adeuring, Wups, that wasn't meant to be there. I'll remove it.
[10:03] <adeuring> gmb: ok, then r=me
[10:06] <gmb> Danke.
[10:20] <jml> grr.
[10:20] <jml> database patches aren't hard, but iterating on them is a pain.
[10:37] <stub> Is the estimated time before a build completes based on all builds, or successful builds?
[10:40] <wgrant> stub: Successful
[10:46] <rick_h> rvba: oops sorry, thought you guys were using 3.5 for some reason.
[11:00] <czajkowski> rick_h: you're up early
[11:03] <rick_h> naw, this is pretty normal for me
[11:04] <rick_h> just normally quiet reading email and MP stuff for the first bit :)
[11:10] <rvba> rick_h: no worries.  Thanks for the review.
[11:23] <jml> may I have another db patch number please?
[11:23] <jml> https://code.launchpad.net/~jml/launchpad/remove-archive-commercial/+merge/109332 is an easy MP, btw.
[11:24] <lifeless_> jml: you can self allocate these days
[11:25] <jml> lifeless_: I'm not in ~launchpad
[11:25] <jml> lifeless_: so I don't have write access to the branch in which patch numbers are allocated.
[11:25] <stub> jml: Gimme a short comment for the log
[11:25] <jml> stub: "Add column to Distribution controlling who can create private PPAs"
[11:26] <stub> jml: 2209-22-1
[11:26] <jml> stub: thanks.
[11:27] <ev> are there any plans to support OAuth 1.0a in launchpadlib? Specifically, oauth_callback.
[11:29] <jml> ev: not that I know of.
[11:30] <ev> jml: cheers
[12:03] <jml> db patch up for review: https://code.launchpad.net/~jml/launchpad/db-distro-level-ppa-privacy/+merge/109336
[12:03] <jml> lifeless_: if you're still kibbitzing, you might be interested in that.
[12:05] <StevenK> jml: You want stub for that
[12:05] <jml> StevenK: sure. but lifeless has been involved in implementation discussions, and might want to be sure that I'm doing what we agreed on, rather than what I think we agreed on
[12:14] <wgrant> jml: You want to talk to sinzui.
[12:14] <wgrant> jml: That DB patch is probably incorrect.
[12:14] <jml> wgrant: I thought I had.
[12:14] <jml> anyway.
[12:14] <jml> wgrant: I'll talk to him again.
[12:14] <wgrant> It's not really anything to do with the distro.
[12:15] <jml> it is now :)
[12:15] <wgrant> How?
[12:16] <wgrant> We generally grant private PPAs to people with commercial subscriptions.
[12:16] <wgrant> Not because they are vaguely related to some distro.
[12:16] <wgrant> It's not a security-sensitive option.
[12:17] <wgrant> So it makes little sense to have it as a distro-delegated operation.
[12:18] <wgrant> Similar to the way we now let anybody with a commercial subscription create a private team.
[12:18] <wgrant> Which implicitly has private PPAs.
[12:19] <jml> Grr.
[12:20] <jml> a) lifeless said "make a new celebrity". I don't know how it became "add a column" in my notes.
[12:21] <jml> b) I emailed launchpad-dev about this 5 weeks ago. No one replied.
[12:23] <wgrant> jml: The thread about the technical bits of protecting it differently in the Zope security model?
[12:24] <wgrant> I don't recall a thread about how it should be modelled.
[12:26] <jml> wgrant: right, the one titled "permissions for creating private objects"
[12:27] <jml> (there was also the email to stakeholders in Jan asking if there'd be "any problems with a PPA having, say, 100,000 subscribers?", but that's a different badger)
[12:27] <jml> sinzui: are you around?
[12:31] <wgrant> jml: Well, that thread didn't go near deciding who can do it. It was around the probably impossible problem of doing it cleanly in the Zope security model.
[12:31] <wgrant> I suspect nobody replied because anybody with an opinion already knows it's hopeless.
[12:32] <jml> wgrant: it opened describing our planned approach. if it was incorrect, that was the time to tell us so.
[12:32] <wgrant> Oh
[12:33] <wgrant> Indeed, the second paragraph of the first email mentions the celebrity approach.
[12:33] <wgrant> Missed it among the other 20 paragraphs of hopeless Zopeness :)
[12:33] <wgrant> Sorry.
[12:33] <wgrant> So, a celebrity is a lightweight and potentially sensible approach for now.
[12:33] <wgrant> A DB column is not.
[12:33] <jml> I put it first (just after what we're trying to achieve) because it was most important (after what we're trying to achieve)
[12:33] <wgrant> But sinzui is Lord of Entitlement.
[12:35] <jml> and is apparently not around.
[12:35] <wgrant> It's not quite 9am for him yet.
[12:35]  * czajkowski peers at wgrant how are you still awake !
[12:36] <wgrant> I'm waiting to talk to a US user who is making several million API requests a day.
[12:37] <czajkowski> wgrant: ah lovely that reminds me
[12:38] <jml> sinzui: I'm getting some food. ping me when you're around and want to talk about creating private PPAs.
[14:22] <cjwatson> wgrant: queue-api: I've converted some of the expensive properties into methods, and arranged to preload everything else.  I think that should address your performance concerns.
[14:23] <cjwatson> (Where "everything else" was just the referencing PackageUpload{Source,Build,Custom} objects.)
[15:09] <jml> sinzui: ping
[15:09] <sinzui> hi jml
[15:09] <jml> sinzui: hi
[15:10] <jml> sinzui: I'd like to change LP so I can create private PPAs without being in commercial_admin
[15:11] <jml> sinzui: consensus from other LP devs is that I should create a new celebrity team that includes everyone who is allowed to do this.
[15:11] <sinzui> jml: Yes, I think that meets your needs.
[15:12] <jml> sinzui: ok. is this in line with other entitlement / privacy work going on?
[15:13] <sinzui> jml: its orthogonal. for entitlement...I think any user/team that maintains a project  with a  commercial subscription can create a p3a.
[15:13] <sinzui> I do not think your rules to maintain the system should be connection to user entitlement
[15:13] <jml> sinzui: well, then why bother making a celebrity team? why don't I just query to see if they maintain a project with a commercial subscription?
[15:14] <jml> sinzui: in fact, why don't I just assume that if they have a private team they can make private PPAs?
[15:14] <sinzui> jml: private teams can only have p3as. maybe I misunderstand you
[15:15] <jml> hmm. thinking...
[15:15] <sinzui> jml: but your statements are true. both will work
[15:16] <sinzui> We do not dismantle private teams when subscriptions expire
[15:16]  * sinzui has no idea to handle that case
[15:16] <jml> sinzui: so, AIUI, currently if you're creating a PPA under a private team you also need to have launchpad.Commercial to make it private.
[15:16] <jml> sinzui: which is silly, I think.
[15:16] <sinzui> oh, jml, there is a method, maybe on personset that will tell you of the user has commercial privs
[15:17] <jml> sinzui: sorry, I shouldn't have said launchpad.Commercial, I should have said "in commercial admin"
[15:17] <sinzui> jml. I hope not...that would be a bug. private teams can not have public ppas
[15:17] <jml> sinzui: well I think it's just that you're not allowed to create any PPA. Let me check
[15:18] <sinzui> that too would be a bug. Any team can have a ppa, private teams are p3a. public teams may only have p3a when someone intervenes.
[15:19] <jml> ok.
[15:19] <jml> I think it is a bug (validatePPA doesn't bother checking the privacy status of the owning team, it just checks admin & commercial_admin)
[15:20] <jml> sinzui: so now we need some way to say who is allowed to create private PPAs on public teams
[15:20] <jml> sinzui: and I guess that can be a celebrity
[15:20] <jml> sinzui: but if there's already some mechanism in Launchpad saying "they've paid for super-powers" then I'd rather use that.
[15:20] <jml> anyway
[15:20] <sinzui> ah, I agree
[15:21] <sinzui> if the user paid for privs, he should see the checkbox or API to make the archive private
[15:21]  * sinzui is still hunting for the method that checks if the user have paid for privs
[15:23] <jml> sinzui: of course, this sort of assumes commercial is all-or-nothing.
[15:23] <jml> sinzui: I guess that's an OK assumption.
[15:23] <sinzui> jml: please assume that
[15:23] <sinzui> we want this very simple for users and us to understand
[15:23] <sinzui> if you find a contradiction, we need to fix it
[15:29] <jml> sinzui: ok, thanks. let me know if you find that API
[15:29] <sinzui> jml, person.checkAllowVisibility() is checks if the user has permission to make things private. It is used to know when to show users the the team visibility field. It would also apply to p3as
[15:34] <jml> sinzui: thanks.
[15:54] <jml> sinzui: please verify: https://code.launchpad.net/~jml/launchpad/db-distro-level-ppa-privacy/+merge/109336/comments/234987
[15:56] <sinzui> jml that is correct with one subtle point you might find in the code...I think we mean users = team admin in some cases.
[15:56] <sinzui> user ~= team admins
[15:56] <sinzui> I do not think team members can create ppas
[15:57] <jml> sinzui: I think they can.
[15:58] <jml> hmm.
[15:58] <jml> maybe not.
[15:59]  * jml can't seem to log in to staging.
[16:09] <jml> sinzui: qastaging uses a different db to prod, right?
[16:10] <sinzui> jml: yes, and it is months older :(
[16:10] <jml> sinzui: you can create PPAs over the API without being a team admin.
[16:10] <jml> sinzui: but not in the UI
[16:10] <sinzui> I think that contradicts ui
[16:10] <sinzui> yep
[16:11] <sinzui> That is a bug, but I do not know how disruptive it is to fix
[16:11] <jml> lbyl bites again.
[16:12] <sinzui> I do not think there is a bug about this. I image it would only exist if teams felt that team privs were being abused
[16:14] <jml> well, it's something we might fix as a follow up, or as a drive-by, but we'll preserve the existing behaviour all things being equal
[16:14] <sinzui> jml: I agree. I would not fix this without an audit of the db
[16:14] <jml> sinzui: how come?
[16:15] <sinzui> Canonical and other commercial users may be taking advantage of this bug.
[16:18] <sinzui> jml: I don't this I could audit this case. We do not know who created the archive. So I think this issue requires a conversation with PES and openstack
[16:19] <jml> sinzui: fair enough.
[16:20] <ev> Before I go too far down the rabbit hole, what's the suggested approach for logging into Launchpad with existing OAuth credentials from a web client?
[16:20] <ev> Should I use Launchpad.login_with and a CredentialStore subclass that provides the already obtained oauth_token and oauth_token_secret?
[16:20] <ev> the existing documentation creates a Launchpad() object from the default constructor with now-missing arguments (https://help.launchpad.net/API/launchpadlib)
[16:21] <ev> err rather: https://help.launchpad.net/API/launchpadlib#Authenticated_access_for_website_integration
[16:25] <sinzui> ev: I am not very familiar with the OAuth code, but I write a lot of API scripts. I think the subclass approach is right so that scripts can be adapted or be written support multiple stores
[16:26] <ev> sinzui: multiple stores? This is only ever going to use cookies. Or have I misread?
[16:26] <ev> back in a bit, it's toasting the new office time
[16:26] <sinzui> ev, As a tester, I would like to switch which cookies I am using.
[16:47] <nigelb> rick_h: Hey, you around?
[16:47] <rick_h> nigelb: sure, what's up?
[16:48] <nigelb> rick_h: Hey, when you worked on SA, what did you guys do for db migration?
[16:48] <rick_h> we used sqlalchemy-migate but I
[16:48] <rick_h> 've got on my todo to port my app over to alembic
[16:48] <nigelb> ah
[16:48] <rick_h> and squash down the migations/etc
[16:50] <nigelb> rick_h: thanks! I look up both of them :)
[16:51] <rick_h> nigelb: yea, definitely suggest alembic, it's done by the SA author so it'll be kept in sync/etc much better
[16:51] <rick_h> nigelb: and he looked at django's south so should be a bit 'learned' from the competition
[16:51] <nigelb> heh
[16:58] <bdmurray> sinzui: so I tried testing via bin/test -vvc --layer=YUI per your suggestion and I keep getting tests timing out
[16:58] <sinzui> That is odd. The tests run in a few minutes for me
[16:58] <sinzui> maybe you are missing dep
[16:58] <rick_h> bdmurray: what's the exact error? It might be that you didn't run the build steps or failed to setup
[16:59] <rick_h> bdmurray: try manually running make jsbuild
[16:59] <bdmurray> AssertionError: JS timed out
[16:59] <sinzui> ah, right, the build files need to be on the filesystem
[16:59] <rick_h> right
[16:59] <ev> sinzui: ah, good point. Thanks for the tips!
[17:01] <sinzui> bdmurray, you can test any YUI module in your browser: xdg-open lib/lp/app/javascript/tests/test_lp_links.html
[17:01] <sinzui> if your browser cannot run the test, the tree was not build right
[17:02] <bdmurray> okay thanks, make jsbuild seems to have helped
[17:02] <rick_h> awesome
[21:17] <lifeless> http://www.infoq.com/presentations/Storm
[21:23] <jelmer> 'morning lifeless
[21:36] <lifeless> hi jelmer