[00:00] <rick_h> wgrant: yea, if we move it to it's own JS file we can just make sure it gets built into launchpad.js and then for the feature flag users make it a seperate request for now
[00:01] <rick_h> wallyworld_: does that make sense? To create a .js file in lib/lp/app/javascript/plain.js or something?
[00:01] <StevenK> wgrant: I have the tests passing, so I can redo the work, and move back to list of tuples or commit and land the use of structure
[00:01] <rick_h> and the builder should pick it up for launchpad.js automatically still
[00:01] <rick_h> since it searches in there right?
[00:01] <StevenK> It runs find over the tree
[00:01] <rick_h> and then we'll manually add the url as a new script tag for feature flag users?
[00:01] <wgrant> StevenK: "redo the work" is about ten lines of code, but I guess it's OK
[00:01] <rick_h> until I can get a chance to run through it and YUI-ize it?
[00:02] <wallyworld_> rick_h: i was just going to wrap in a CDATA tag for now - we already do that all over the place
[00:02] <wallyworld_> essentially a 2 line change to the mp
[00:02] <rick_h> wallyworld_: ok then. I've never done that so that was the less comfy way for me
[00:02] <rick_h> wallyworld_: but awesome, works for me
[00:02] <wgrant> Make sure that the CDATA tags are commented.
[00:02] <wgrant> Or IE's JS parser will choke
[00:02] <wallyworld_> of course
[00:02] <wallyworld_> i cut n pasted :-)
[00:03] <wgrant> The web needs to die.
[00:03] <wallyworld_> from elsewhere
[00:03] <wallyworld_> no, IE needs to die
[00:03] <wgrant> Holy shit
[00:03] <wgrant> IE9 supports XHTML
[00:03] <rick_h> yea, IE9 is the first real browser in a long time
[00:04] <rick_h> IE10 actually might make me not hate it..but until then
[00:04] <wgrant> Only 12 years late :)
[00:04] <rick_h> IE9 is still missing html5 history/geo location/etc that everyone else has now
[00:05] <rick_h> wallyworld_: ok, so looking at: http://yuilibrary.com/yui/docs/api/files/yui_js_yui-log.js.html#l36
[00:05] <StevenK> wgrant: Compilation failed', "zope.tal.taldefs.TALError: invalid syntax (<string>, line 1) in expression u'python:${tag/1}', at line 148, column 15"
[00:05] <rick_h> the logging is looking for a setting in Y.config.debug = false
[00:05] <wgrant> StevenK: python: tag[1]
[00:05] <StevenK> Sigh, TALES.
[00:06] <wgrant> you can hardly say python: and then give it TALES instead.
[00:06] <wallyworld_> rick_h: that's for the src filters - include and excluded modules
[00:06] <rick_h> wallyworld_: and looking at how the YUI instance is setup, if you were to add config: { debug: false } to the list in YUI.GlobalConfig that would work
[00:06] <StevenK> wgrant: Silence. :-P
[00:06] <rick_h> wallyworld_: it wraps the whole thing
[00:06] <rick_h> if the debug is false it bypasses all logging
[00:07] <wallyworld_> rick_h: but we still want errors logged in prod
[00:07] <rick_h> wallyworld_: the whole method does nothing but return the Y instance at the end
[00:07] <rick_h> wallyworld_: ok, then we'll have to set the config.logExclude filters for warning?
[00:08] <rick_h> nvm, those are for the source, we need on log level, looking me
[00:08] <wallyworld_> rick_h: my understanding is that the include/exclude filetrs are for modules
[00:08] <rick_h> looking more :/ can't type
[00:08] <wallyworld_> afaict, we need logLevel but that's not a global option sadly
[00:09] <rick_h> wallyworld_: the problem is that's on the console module
[00:09] <rick_h> we want to touch the raw log included in yui.js
[00:09] <rick_h> they're different, the loglevel would do us no good here
[00:09] <wallyworld_> yep :-(
[00:10] <wallyworld_> that's the conclusion i camr to as well
[00:10] <rick_h> wallyworld_: I still say we turn off debug. This is the YUI log module. Any JS errors are exceptions that go around this
[00:10] <StevenK> wgrant: I had to set tal:content to python: tag[0] too, but I think I have it working.
[00:10] <rick_h> wallyworld_: the only errors this would eat are places where we willingly do Y.log.error()
[00:10] <wgrant> StevenK: Of course
[00:10] <wallyworld_> rick_h: sounds like a reasonable plan i think
[00:11] <rick_h> or sorry, Y.log(msg, "error")
[00:11] <wallyworld_> yeah, knew what you meant :-)
[00:11] <StevenK> wgrant: So I have your blessing to toss it at ec2 and find something else to do?
[00:12] <wgrant> StevenK: If it doesn't use structure, I have no objection.
[00:12] <wgrant> Because the worst you can do is break the links :)
[00:13] <rick_h> wallyworld_: so I'm not sure which of these you need: http://paste.mitechie.com/show/505/
[00:13] <wallyworld_> rick_h: i knew 30 minutes ago when i read the docs :-)
[00:13] <wallyworld_> i'll sort it
[00:14] <wallyworld_> short term memory is broken, brain too full, it hurts to think
[00:14] <rick_h> wallyworld_: I *think* it's the first but not 100% sure
[00:14] <rick_h> but anyway, if it fails jsut a heads up you might need the other
[00:15] <wallyworld_> np, will find out in about 3 minutes
[00:18] <wallyworld_> rick_h: it's just "debug: true|false"
[00:18] <rick_h> wallyworld_: yea, the check in the code is if (!c.debug)
[00:18] <rick_h> sorry, if (c.debug)
[00:25]  * StevenK has been seeing some wierd behaviour today
[00:26] <StevenK> Refreshing MP pages "Shows transferring data" for a good long while until I get bored and stop it
[00:28] <wallyworld_> StevenK: bazaar.lp.net was sloooow yesterday
[00:28] <wallyworld_> timed out a lot
[00:28] <wallyworld_> rick_h: now all i need is deryck to come back online and +1 the fucker :-)
[00:29] <StevenK> rick_h: WRT to your convoy MP, I was expecting we'd have /+combo/r14335/ and then something like /+combo/r14436/ ?
[00:30]  * StevenK stabs the messaging indicator more
[00:30] <wallyworld_> StevenK: the use-convoy mp is/will be linked against 2 bugs. if i land with --incr, both bugs will remain in progress, right?
[00:30] <StevenK> I believe so
[00:31] <wallyworld_> that makes me sad, since one will be fixed
[00:31] <wallyworld_> i guess we can close manually
[00:32] <StevenK> Right, we don't really deal with that case
[00:37] <poolie> i can do recipe builds from private branches, right?
[00:38] <StevenK> You can not.
[00:38] <poolie> ah, foo, i thought i'd seen it
[00:38] <poolie> maybe i was thinking of private ppas
[00:38] <jelmer> StevenK: do you remember what prevents them?
[00:38] <wgrant> jelmer: Builders don't have SSH keys
[00:39] <poolie> was this connected to mwh's thing of allowing ssh with a limited-time token?
[00:39] <wgrant> Precisely.
[00:39] <jelmer> wgrant: oh, right
[00:40] <wallyworld_> jelmer: sorry to bother you - is there anything i can do to get lp-land etc working in precise?
[00:42] <jelmer> wallyworld_: can you tell me what package version of bzr-pqm you have installed?
[00:43] <wgrant> wallyworld_: Do you really have to conflate those two changes into a single branch?
[00:43] <wallyworld_> jelmer: 1.40-bzr80-1
[00:43] <wgrant> wallyworld_: One is almost guaranteed to include regressions and need rolling back at least twice, while the other is a critical regression fix.
[00:43] <wallyworld_> wgrant: i could land in devel but then there will be merge conflicts for the other one
[00:44] <wgrant> Ah
[00:44] <wallyworld_> "at least twice" - you so such a pessimist
[00:44] <wgrant> How safe is use-convoy with the flag off?
[00:44] <wgrant> This sort of stuff has an exceptionally bad track record.
[00:44] <wallyworld_> it uses ye old launchpad.js as before
[00:44] <wgrant> Sure, but there are presumably some changes.
[00:44] <wgrant> And none of this has tests.
[00:45] <wallyworld_> minor ones - mainly going to multiple YUI instances instead of a single instance
[00:45] <wallyworld_> all the yui tests pass
[00:46] <wgrant> The YUI tests don't use launchpad.js
[00:47] <wallyworld_> true, but i diffed launchpad.js from trunk and it was the same
[00:47] <wallyworld_> as the branch
[00:47] <jelmer> wallyworld_: the easiest thing to do is probably to run "bzr branch lp:bzr-pqm ~/.bazaar/plugins/pqm"
[00:47] <wallyworld_> accounting for expected skew
[00:51] <wallyworld_> jelmer: thanks! looks ok. i'll know for sure when i land the next branch
[00:52] <wgrant> wallyworld_: Ah, OK
[00:52] <wgrant> I guess it's somewhat reasonable, then :)
[00:53] <wallyworld_> wgrant: i bloody hope so at least. i've done a lot of smoke testing.
[00:53] <wallyworld_> but lp is soooo big it's impossible to touch everything
[00:53] <rick_h> StevenK: no, the other way around. /$revno/+combo
[00:53] <rick_h> that way we can have as many paths as we want in there to start
[00:54] <wallyworld_> wgrant: if my optimisim ^H^H^H^H^H blind hope proves false, i'll do a separate fix for the bug straight away
[00:54] <rick_h> StevenK: so if the files were in /var/convoy/build/js we could set the root to /var/convoy and make the app provide /build/js/$revno/+combo as the url
[00:55] <wgrant> wallyworld_: Sounds good.
[01:03] <StevenK> rick_h: Really? The icing stuff is icing/<revno>
[01:03] <StevenK> rick_h: And lifeless said he wanted it like it
[01:04] <rick_h> StevenK: right, well I don't think the order matters. In this way it's easier for me to regex since I can basically match anything up until the last item and make that a path (supporting multiple levels if required) and combo or +combo don't matter so we don't have to deal with changing upstreams usage
[01:05] <StevenK> rick_h: But it does matter
[01:05] <rick_h> StevenK: I think this gets the effect we want (changing the paths at run time based on the LP instance) without making too many changes
[01:05] <rick_h> why?
[01:05] <StevenK> rick_h: Since /+combo is proxied off in the apache config
[01:05] <StevenK> So the URL has to *start* with /+combo
[01:05] <rick_h> right, but we can't change that check to combo$?
[01:06] <StevenK> I really don't want to do regex matches in the apache config
[01:06] <rick_h> ok, shoot me a couple of urls we want to match and I'll see if I can change the regex to support what we want and what U1 and Landscape are doing at the same time
[01:07] <rick_h> I didn't realize we had the limitation so I was merely trying to add directory support
[01:07] <StevenK> rick_h: So, I'd make it configurable -- turn it on with a parameter in combo_app()
[01:08] <rick_h> StevenK: so you're thinking a config of "allow_paths" or something that turns on/off grabbing path info from the url?
[01:08] <rick_h> and then just if set do it the LP way?
[01:09] <StevenK> rick_h: And I was expecting something like /+combo/r12456/yui/yui/yui-min.js or /+combo/r14858/lp/registry/timeline.js
[01:09] <StevenK> rick_h: Right, exactly.
[01:09] <rick_h> ok, so I want to match /(toss)/(some)/(path)/(hints)/?... then right?
[01:09] <rick_h> ok, I think I can reverse the regex for that in convoy. I'll update the tests and push a new one tomorrow then
[01:11] <StevenK> Right, so what you want to do for /+combo/r12456/?yui/yui/yui-min.js is actually return <root>/r12456/yui/yui/yui-min.js. Not really important, but I'd prefer the comment showing the filename didn't include the revno dir
[01:12] <rick_h> right, all good
[01:12] <rick_h> I follow, sorry, just was thinking normal url with path first, file end
[01:12] <rick_h> should be an easy adjustment
[01:24] <wgrant> sinzui: Still around to discuss the SSo issue a bit?
[01:26] <sinzui> I am about
[01:26] <wgrant> sinzui: Do you think the emailaddress-stealing code is still useful?
[01:27] <sinzui> wgrant, I do. I recall users being very confused about the email mismatch
[01:28] <sinzui> wgrant, but I really think the issue is that users expect synced email addresss to be synced
[01:28] <wgrant> sinzui: More confusing than when they get logged into a different account, and then can't find their original account because it no longer has an email address to search by?
[01:28] <sinzui> wgrant, If the l.lp.net did not lie about what/who it was, the sync issue might be mitigated
[01:30] <wgrant> sinzui: We should certainly inform affected users that something is up.
[01:30] <sinzui> wgrant, yes.
[01:30] <wgrant> But I think that automatically "fixing" it is harmful.
[01:30] <wgrant> Because in most cases the second SSO account is mistake.
[01:31] <wgrant> And transferring the address to the second LP account just exacerbates the mistake.
[01:31] <wgrant> I think we should rather warn on login.
[01:31] <wgrant> That $otherperson has the primary address, see this wiki page for details.
[01:32] <sinzui> We could explain we think identities are mismatched. We could let the user choose the transfer the email address (without email confirmation)
[01:32] <wgrant> Right.
[01:32] <wgrant> In fact, I was thinking that we could only really do this with addNotification.
[01:32] <wgrant> But +openid-callback could prompt directly.
[01:32] <wgrant> Add an extra step to the login.
[01:33] <sinzui> Do we know enough info when this happens to advise the user to go back to SSO and sort the matter out? Do we know enough to tell the user about the two identities in Lp that are criss-crossed?
[01:34] <sinzui> We sort of know in the case of Ubuntu. I do not think so in the case of another SSO provider
[01:34] <wgrant> We can identify the other persoin in LP.
[01:34] <wgrant> In the case of another SSO provider it doesn't matter.
[01:34] <wgrant> Because we don't trust their email addresses.
[01:34] <wgrant> So we *can't* do reassignment.
[01:34] <sinzui> Well, Do I (me) really trust Ubuntu SSO at the moment.
[01:34] <wgrant> We want to reassignment purely because login.launchpad.net confuses people by looking like Launchpad.
[01:35] <wgrant> Heh
[01:35] <wgrant> No.
[01:35]  * sinzui looks if he can delete email address now
[01:35] <wgrant> You can now.
[01:35] <wgrant> For a few months now.
[01:35] <StevenK> kill-bugtask-launchbag-with-fire => devel  [FAILED]   (up for 1:38:25) i-568d8234
[01:35] <StevenK> :-(
[01:35] <wgrant> StevenK: Which test?
[01:36] <StevenK> I haven't looked yet
[01:36] <wgrant> sinzui: So, I think that if an identifier is known we should trust it. If the email address isn't known by Launchpad, we add it to their person. If it *is* known by Launchpad but is on another account, we log them in but tell tham what's going on and how to fix it.
[01:36] <wgrant> sinzui: They (and we) can do the proper fix then.
[01:37] <wgrant> Because we can present the primary email addresses of both people.
[01:37] <sinzui> Ubuntu is almost as bad as google. If I leave Canonical, my primary identity is compromised. People leave their employers more often then they change the personal addresses.
[01:37] <wgrant> Whereas now we silently steal the conflicted primary email address, making it very hard to work out what the original state was, and how to recover.
[01:37] <sinzui> I still do not trust
[01:38] <wgrant> This also moves us towards a workflow for external providers.
[01:38] <wgrant> Where logging in will create an invalid person, because it has no verified email addresses.
[01:39] <wgrant> This same post-login workflow process could be used to verify an address.
[01:39] <sinzui> wgrant: +1. Fixing bad data is always harder than letting a user know that something is wrong
[01:39] <wgrant> sinzui: Right. And this situation only arises when a mistake has already been made, because just about nobody has a reason to have two SSO accounts.
[01:41] <sinzui> wgrant, the mistake was ours I think. We made no distinction between imported identities and those users claimed and were active
[01:42] <sinzui> SSO should purge ever address that that was not active when received from Lp and only accept those that are given my users
[01:44] <wgrant> sinzui: Until last week SSO still had emailaddresses for Launchpad teams.
[01:44] <wgrant> But we deleted those.
[01:44] <wgrant> I don't know if it still has NEW ones.
[01:45] <sinzui> including @lists.launchpad.net? That is an obvious imposability
[01:45] <wgrant> I've endeavoured to understand the situations in both DBs and codebases.
[01:45] <wgrant> But it is complicated.
[01:45] <wgrant> I believe all @lists.launchpad.net addresses should be gone from SSO's DB now, unless they are associated with a proper account.
[01:45] <wgrant> Which is a bit crazy, but possible I suppose.
[01:46] <wallyworld_> rick_h: StevenK: use-combo is in ec2 :-)
[01:47] <rick_h> yay!!!
[01:51] <sinzui> wgrant, when Lp sees a mismatched email address. What can Lp do for the user? Transfer the email? I think Lp should suggest a merge instead. Since SSO does not have merging, I do not think we can explain how to make the identities the same if sso has 2+ identities for the user
[01:51] <wgrant> sinzui: Right, I'm not sure.
[01:51] <wgrant> sinzui: I've also just realised a complication.
[01:51] <sinzui> Its a lost of email deletion and reclaiming I suspect
[01:52] <wgrant> sinzui: If we don't steal the address, how do we deal with reactivating a deactivated account?
[01:52] <wgrant> We don't have a preferredemail :/
[01:52] <wgrant> So I have a deactivated account, and log in with its OpenID, but the SSO preferredemail is on a different account.
[01:53] <wgrant> The only way out is a logintoken.
[01:53] <wgrant> Either to do the merge, or to validate and select a new preferredemail.
[01:54] <sinzui> well, I think you are proposing a fix to an unreported bug. User complain we did not deactivate their account because they can login again. If Lp asked the user he wants to proceed, he would not compain
[01:54] <wgrant> It gets even more complicated when we consider SCA
[01:54] <wgrant> Which relies on the implicit reactivation behaviour.
[01:54] <wgrant> And cannot have user interaction.
[01:55]  * wgrant dies violently.
[01:56] <sinzui> You mean we make a Frankenstein identity from other identities, log the user in and say...Your Alive! Click here to die if you do not want to be a gestalt
[01:57] <wgrant> Until last week the only way getOrCreateByOpenID could fail was if the account was suspended.
[01:57] <wgrant> Currently it will fail if the provided email is owned by a team.
[01:57] <wgrant> And ideally it would not return a valid person if it had to reactivate.
[01:57] <wgrant> But SCA assumes it does.
[01:58] <wgrant> And SCA involves money :)
[01:58] <sinzui> I am not keen on suspending another 3000 spam profiles...screw SCA
[01:59] <wgrant> I wonder if lifeless has jurisdiction over CA
[01:59] <wgrant> Which owns SCA nowadays.
[01:59] <wgrant> Ah, no, they're under OS
[01:59] <wgrant> Sad.
[02:03] <wgrant> I guess SCA is pretty similar to SSO
[02:03] <wgrant> In that they both touch Launchpad in completely inappropriate ways.
[02:03] <wgrant> I'm just less sure about how to fix SCA.
[02:03] <wgrant> Because it relies on being able to obtain a Launchpad person for any unsuspecting SSO account.
[02:04] <sinzui> Do we need to fix it now. Do users want us to fix it?
[02:04] <wgrant> The problem is I broke it last week.
[02:04] <wgrant> And fixing this cleanly will break it further.
[02:04] <sinzui> Telling users we can fix SCA by doing what we do now and locking you out of part of your identity
[02:04] <wgrant> Perhaps I should just restore the old behaviour of stealing the team address, however distasteful that may be.
[02:05] <wgrant> This doesn't fix any of the other problems, though.
[02:05] <wgrant> SCA prevents us from fixing the corruption issues :(
[02:07] <wgrant> Although bug #901336 is encouraging.
[02:07] <wgrant> It already breaks in some cases.
[02:07] <wgrant> So maybe we can break it more and nobody will complain :)
[02:07] <sinzui> Stealing a team address is very bad. We know every team address is a one that is being used. We delete any team address that the user does not indicate is being used
[02:09] <sinzui> Lp has to tell the user that the address belongs to a  team. The options are to remove the address (confirm via email to do it), login as the owner and delete the team, or go that to SSO and remove the badness
[02:09] <wgrant> Sure.
[02:09] <wgrant> That's clearly the correct solution.
[02:09] <wgrant> But SCA relies on being able to get or create an LP person for any SSO account.
[02:09] <wgrant> Without user interaction.
[02:10] <wgrant> We probably want to decide to break that.
[02:11] <sinzui> I do not think SCA advocates identity theft. Surely they have some guidelines for when to not honour the operation
[02:12] <wgrant> The XML-RPC API that they use pretty much just calls getOrCreateByOpenIDIdentifier directly.
[02:12] <wgrant> And the only way that could fail was in the case of a suspended account.
[02:27] <wgrant> Although I guess if they getOrCreate the person early, they can fail before the transaction.
[02:29] <wgrant> sinzui: So, I think for now I will fix it to steal team addresses again. But I will obtain the SCA code and talk to CA people about how it can do better.
[02:30] <sinzui> okay
[02:30] <wgrant> The quick fix is probably just to attempt to get the person right before payment.
[02:30] <wgrant> That way there's only a small race, which probably already exists.
[02:36] <wallyworld> StevenK: bollocks. we need to get a new ec2 image with convoy packaged or else make fails
[02:36] <wgrant> wallyworld: Or just add it to launchpad-developer-dependencies, and ec2 should auto-install it.
[02:37] <wallyworld> wgrant: of course, much easier
[02:37] <wallyworld> StevenK: ^^^^ can you do that? i have no idea how and want to get this fucker though ec2
[02:38] <wallyworld> actually, i think there's already a partial branch for it
[02:38] <wgrant> Ah, buildbot will need a manual upgrade too, actually.
[02:42] <wallyworld> how do we do that?
[02:43] <wgrant> webops :)
[02:43] <wallyworld> thought that may be the case
[02:44] <wallyworld> i'll wait for StevenK to pop up and we can get it sorted in one go
[02:44] <wgrant> Oh, blah.
[02:44] <wgrant> That means convoy will need catification
[02:44] <wallyworld> you mean whiskers and a saucer of milk?
[02:44] <wallyworld> :-)
[02:44] <wgrant> Close enough.
[03:00] <StevenK> Oh, right
[03:01] <StevenK> wgrant: You don't like that idea?
[03:08] <StevenK> wallyworld: O hai -- https://code.launchpad.net/~stevenk/meta-lp-deps/use-convoy/+merge/89174
[03:08] <wallyworld> StevenK: you want a review?
[03:09] <StevenK> wallyworld: I do, yes.
[03:09] <StevenK> wallyworld: I have to wait for convoy to publish in the LP PPA anyway.
[03:09] <wallyworld> looks ok to my untrained eye, will _1
[03:09] <wallyworld> +1
[03:10] <wallyworld> StevenK: so what's the next step? looks like i can't ec2 land today?
[03:10] <StevenK> You could, but it would then fail buildbot
[03:11] <wallyworld> bollocks
[03:11]  * StevenK sees if convoy is installable from the PPA
[03:11] <wallyworld> so we could get your ppa installed on bb?
[03:12] <StevenK> Only if you want a GSA to visit you personally with some sharp implements
[03:12] <StevenK> wallyworld: Jump on mumble, and I'll outline it
[03:13] <wallyworld> StevenK: ok. i may have to reboot so if i drop off, i'll be back in a bit
[03:13] <wgrant> wallyworld: So, I'd suggest ec2 testing today
[03:13] <wgrant> If it works, then get convoy into cat and onto bb
[03:13]  * StevenK glares at mumble
[03:14] <StevenK> 3 copies?!
[03:14]  * StevenK wields kill
[03:14] <lifeless> not to be landing such think on friday
[03:31] <StevenK> lifeless: WCPGW!
[03:57] <wallyworld> sinzui: +1 on that mp if you want to send it to ec2 before you go to bed
[04:00] <StevenK> wallyworld: meta-lp-deps changes merged, waiting for the recipe builds -- they should kick off automatically in the next 1-2 minutes
[04:00] <wallyworld> \o/ thanks :-)
[04:01] <wgrant> StevenK: Did you just make them uninstallable in the DC? :)
[04:06] <StevenK> wgrant: Huh?
[04:07] <StevenK> wgrant: But buildbot doesn't update from the PPA?
[04:07] <wgrant> StevenK: No.
[04:07] <wgrant> StevenK: But making the dependency packages uninstallable on buildbot is pretty unpleasant.
[04:09] <StevenK> wgrant: I was of the opinion that the dependencies packages had to get copied to CAT
[04:09] <wgrant> Ah, true.
[04:18] <StevenK> Oh, damn it.
[04:19] <StevenK> My launchbag branch failed with a message so large my mail server rejected it.
[04:19] <nigelb> lol
[04:19] <StevenK> And I think the instance has just killed itself.
[04:28] <wgrant> StevenK: Locally run bin/test -cvvt lp.bugs?
[04:28] <StevenK> Yes, I was planning on
[04:35] <StevenK> wallyworld: Right, meta package updated.
[04:35] <StevenK> Strangely, apache2 is a Suggests of developer-dependencies
[04:35] <StevenK> It's forcibly installed by rf-setup
[04:35] <wallyworld> wonder why
[04:36] <StevenK> Because it's not required to run the tests
[04:36] <wallyworld> ok
[04:36] <StevenK> So people will have to install libapache-mod-wsgi when they want to test the combo loader
[04:37] <StevenK> wallyworld: In either case, you can toss your branch at ec2 test
[04:37] <StevenK> Sigh, libapache2-mod-wsgi
[04:37]  * wallyworld tosses
[05:03] <StevenK> wallyworld: It should have built, can you run 'bin/ec2 ls --show-urls' and see what the log shows
[05:04] <wallyworld> StevenK: ah, bollocks. i didn't look back at the console after i ran and there was an error. trying again
[05:04] <StevenK> What error?
[05:05] <wallyworld> i pasted in the mp url instead of the branch
[05:12]  * wgrant tries not to be sick.
[05:14] <StevenK> wgrant: Hmm?
[05:16] <wgrant> StevenK: https://bugs.launchpad.net/launchpad/+bug/556680/comments/5 and https://bugs.launchpad.net/launchpad/+bug/881019/comments/15
[05:16] <wgrant> Beware, there is OpenID.
[05:19] <StevenK> The novel you posted to bug 556680 is impressive.
[05:21] <wallyworld> wgrant: can has quick review? https://code.launchpad.net/~wallyworld/launchpad/sortable-dates-fix-918892/+merge/89380
[05:22] <wgrant> wallyworld: Ah, heh.
[05:22] <wgrant> r=me
[05:23] <StevenK> wallyworld: I won't ask how long *that* took to track down.
[05:23] <wallyworld> thanks. took me ages to find the problem. seems really simple now
[05:23] <wallyworld> about 4 #@%@^ hours
[05:25] <wallyworld> my brain hurts sooooo much today. this week it has done a lot of cogitating
[05:26] <wallyworld> wgrant: could use lp-land for me? pretty please. ec2 land now works but lp-land still is broken
[05:26] <wgrant> Good to see you living dangerously :)
[05:27] <wallyworld> well, there are no tests for it
[05:27] <wgrant> Heh
[05:27] <wallyworld> and it's broken anyhow
[05:28] <wgrant> Submitted
[05:28] <huwshimi> wallyworld: What's wrong with you lp-land?
[05:29] <huwshimi> *your
[05:29] <wallyworld> huwshimi: an api change in precise
[05:29] <wallyworld> one of the bzr libs
[05:29] <huwshimi> wallyworld: Oh, awesome
[05:30] <wallyworld> various get_foo methods replaces by get('foo'), or something like that
[05:30] <wallyworld> it's known
[05:42] <StevenK> wgrant: Total: 3223 tests, 112 failures, 99 errors in 56 minutes 38.785 seconds.
[05:43] <StevenK> I think I see why the failure mail was 15MiB.
[05:43] <StevenK> wallyworld: I wonder if pqm-submit works for you
[05:44] <wallyworld> StevenK: use-convoy => devel  [FAILED]   (up for 0:37:03) i-e81c158a.... waiting for the email to arrive to see what happened
[05:44] <StevenK> That will take another 3+ hours
[05:44] <StevenK> You can add --show-urls and open the URL it shows
[05:45] <wallyworld> StevenK: actually, i updated bzr-pqm earlier today so perhaps it will work
[05:46] <wallyworld> StevenK: failures due to doc test errors. extra lines like "import site' failed; use -v for traceback"
[05:47] <StevenK> Hm
[05:47] <StevenK> You may have to run them locally
[05:47] <wallyworld> so with that extra output, most all doc tests break. but other tests appear fine
[05:47] <wgrant> It's of course possible that ec2 only runs apt-get upgrade, so it won't have installed convoy...
[05:48] <wallyworld> how is import site related to convoy being installed?
[05:48] <wgrant> Is there much else it could be related to?
[05:48] <wgrant> They're not obviously related, but it's the only thing that comes to mind.
[05:48] <wallyworld> no, but i don't get the correlation
[05:48] <wgrant> site overrides our pythonpath and various other evil things.
[05:49] <StevenK> wgrant: The make was successful, so it has
[05:49] <wgrant> Hmm
[05:50] <StevenK> upgrade will install new dependencies of already installed packages that are to be upgraded
[05:50] <wgrant> Except sometimes it doesn't.
[05:50] <StevenK> Depends on the situation
[05:51] <wallyworld> yes, without convoy installed, make failed
[05:51] <StevenK> apt has some funny ideas
[05:51] <wgrant> Frequently eg. linux-generic gets held back, despite just requiring installation of new ABIed packages.
[05:51] <wallyworld> i tried locally a test and i got a different error
[05:51] <wallyworld> 'import sitecustomize' failed; use -v for traceback
[05:52] <wallyworld> similar but different
[05:53] <StevenK> wgrant: ec2 does aptitude update and aptitude -y full-upgrade
[05:53] <StevenK> Yay for pointlessly different package resolution
[05:54] <wallyworld> i get 'import sitecustomize' failed; use -v for traceback whenever devel/bin/py runs
[05:54] <wallyworld> everything that doesn;t try and match text against stdout like doc tests do seems to work
[05:55] <StevenK> Let's delete all doctests.
[05:55] <StevenK> I can't see anyone disagreeing with that plan.
[05:55] <wallyworld> i'd still like to know wtf is going on
[05:56] <wallyworld> doctests are useful if they are used as doco plus examples to show how to use the apis
[05:56] <wgrant> use -v for traceback :)
[05:58] <StevenK>     self.user = request.user
[05:58] <StevenK> AttributeError: 'LaunchpadTestRequest' object has no attribute 'user'
[05:58] <StevenK> RARGH!
[05:58] <StevenK> Hmm
[05:58] <StevenK> request.principal ?
[05:59] <wallyworld> python 2.7 related error it seems https://pastebin.canonical.com/58488/
[06:00] <wgrant> StevenK: request.user should be right, but maybe LTR doesn't provide it.
[06:00] <StevenK> It would seem it does.
[06:00] <StevenK> Er, does not
[06:04] <StevenK> wgrant: lbr.user == AttributeError: 'LaunchpadBrowserRequest' object has no attribute 'user' / lbr.principal == None
[06:09]  * StevenK shakes his fist at LaunchBag
[07:50]  * StevenK is concerned that we don't any production OOPS report from today.
[07:52] <wgrant> StevenK: postgres was restarted for an upgrade, so it's not entirely surprising.
[07:52] <StevenK> Ah
[07:52] <wgrant> The amqp2disks died, but I restarted them.
[07:53] <StevenK> wgrant: http://www.flickr.com/photos/telekon/6718360105/
[08:59] <adeuring> good morning
[09:08] <danhg> Morning guys
[11:32] <rick_h> morning
[12:25] <jml> Does anyone know where lifeless's list of architecture values lives on the web? (ISTR "visible" and "fast" being two of them)
[12:28] <cjwatson> jml: http://launchpad.readthedocs.org/en/latest/values.html ?
[12:29] <cjwatson> or http://launchpad.readthedocs.org/en/latest/strategy.html
[12:29] <cjwatson> (https://dev.launchpad.net/ -> "Documentation in the Launchpad tree" -> ...)
[12:29] <rick_h> we've got RTD? didn't know that
[12:29] <jml> cjwatson: thanks, but those are more functional / user-level values. (Also, those are the ones I wrote :))
[12:29] <jml> hmm. actually, I think it's in a slide somewhere.
[12:29] <cjwatson> ah, heh
[12:30]  * cjwatson goes to teach granny to suck eggs, too
[12:30] <jml> cjwatson: heh heh
[12:31] <jml> rick_h: yeah, I set it up in a fit of effort for making the codebase more approachable through more navigable docs
[12:32] <rick_h> cool, /me markes the buildout doc as must read sometime today
[12:32] <jml> I guess people still rely more on code and the oral tradition.
[12:32] <jml> rick_h: I'm biased, but the README is pretty fun.
[12:33] <jml> found it: https://docs.google.com/a/canonical.com/present/view?id=0AR8DGdpwOJuiZGdwZGNmbjlfN2t2ZHBxanhx&hl=en_US
[12:33] <cjwatson> Code gets less out of date :-/
[12:33] <jml> oh huh, https://dev.launchpad.net/ArchitectureGuide/
[12:34] <rick_h> yea, and code you can test/run with. Often I find I end up chasing one doc to another for hours.
[12:34] <rick_h> it's like getting sucked into wikipedia
[12:34] <jml> cjwatson: yeah.
[12:34] <jml> actually, well, sort of.
[12:35] <jml> one thing LP suffers from is that the way of doing things changes, but not all of the instances of the old way are updated
[12:35] <jml> or are partially updated
[12:35] <jml> and then when you go to use some existing code as a reference, you have little clue as to whether it's a good quality reference.
[12:35] <rick_h> +1 to that
[12:36] <cjwatson> Yep.  The existence of deprecated functions that have no external API ...
[12:36] <cjwatson> JFD(elete)I
[12:36] <jml> or multiple variant spellings of displayname, date_created etc.
[12:37] <jml> Oh how I'd love to go to Launchpad with a sharp knife and a clean conscience.
[12:38] <cjwatson> the piles of sqlobject stuff still around are particularly confusing for this newcomer
[12:39] <jml> this old timer too.
[12:48] <StevenK> cjwatson: I get quite scared about moving some of the Soyuz stuff from SQLObject to Storm since some of the queries are on a knife-edge as it is ...
[12:52] <bigjools> yeah, there's loads of preloads going on in the old-style queries that will be affecting code used in completely different placves
[12:52] <bigjools> bit of a shambles
[12:52] <wgrant> A lot of it is already so slow that it doesn't really matter, though.
[12:52] <wgrant> It's not *that* bad.
[12:54] <jml> StevenK: I guess if you're reluctant to do something useful because of uncertain negative consequences, then the thing to do is find some way of safe experimentation.
[12:56] <jml> StevenK: there's such a thing as paralysis from lack of analysis
[12:57] <StevenK> On the other hand, I did just change a query in BPPH from SQLObject to Storm and due to having a bit of denormalised data and a new index, it dropped from 6000ms to about 30.
[12:59] <jml> woot
[13:00] <Laney> What's the right way to run doctests? I'm just getting NameError for everything.
[13:00] <rick_h> Laney: which doctests are you trying to run? the filename?
[13:00] <Laney> bin/test -cvv -t soyuz/doc/publishing.txt
[13:01] <StevenK> No path
[13:01] <StevenK> Just publishing.txt
[13:01] <Laney> ah
[13:01] <wgrant> Well
[13:01] <wgrant> -t takes a regex
[13:01] <Laney> yeah, still fails
[13:02] <wgrant> what's the error?
[13:02] <Laney> if HTTP was working I could show you ...
[13:03] <Laney> wgrant: http://paste.debian.net/152951/ and others like it
[13:03] <wgrant> Those are cascading from a real failure at the top.
[13:03] <Laney> do I need to set up the database somehow?
[13:03] <wgrant> You need to find the first error.
[13:03] <wgrant> Setting copied_binary failed at some point.
[13:03] <Laney> 1s.
[13:03] <wgrant> doctests are a bit awesome like that -1
[13:03] <wgrant> Er
[13:04] <wgrant> The -1 option used to show only the first error.
[13:04] <wgrant> But that broke a year or so ago :(
[13:04] <jml> buh bow
[13:04] <Laney> yes, yes, I see it now. Somewhat buried.
[13:05] <jml> for the parallel testing stuff, I guess you're going to stick w/ z.testing?
[13:05] <wgrant> jml: Yep
[13:05] <Laney> do they use a different database?
[13:05] <wgrant> jml: Just with testr + LXC on top of it
[13:05] <wgrant> Laney: Each time the testrunner starts it creates a fresh DB from launchpad_ftest_template, named launchpad_ftest_$PID
[13:06] <Laney> right, so I need to patch the template. ta.
[13:06] <rick_h> anyone give me a hint what I'm looking for with this question in #launchpad? http://paste.mitechie.com/show/507/
[13:06] <rick_h> the thing says "published 17hrs ago" and I'm not seeing a queue or something else that's pending
[13:06] <jml> wgrant: makes sense. still, would be nice to ditch zope.testing, or at least be not using a fork of an old version.
[13:16] <wgrant> jml: Indeed.
[13:28] <Laney> https://code.launchpad.net/~laney/launchpad/add-sponsor-field-to-spph just fixed the test failures — stupid omission — and would appreciate another go at ec2
[13:32]  * cjwatson finds
[13:32] <cjwatson>     # A helper for the common case.  IWBNI we had Python 2.5 because
[13:32] <cjwatson>     # then we could just use functools.partial().
[13:33] <rick_h> Laney: sure thing, will fire it off
[13:33] <wgrant> cjwatson: At least it's only in a test.
[13:35] <cjwatson> It's OK to rely on switch_dbuser committing the transaction, right?
[13:36] <Laney> rick_h: thanks
[13:39] <wgrant> cjwatson: Yes. But you should normally be using 'with dbuser', probably.
[13:39] <wgrant> cjwatson: You can also use it from FunctionalLayer now, rather than just ZopelessLayer.
[13:40] <wgrant> You'll still see some tests around saying they're ZopelessLayer so they can change DB users, but I fixed that a few months ago.
[13:42] <cjwatson> wgrant: Right, I just found an old thing that said it had to commit because switching DB users did transaction.abort.  I'll kill that.
[13:42] <wgrant> cjwatson: That used to be the case, but 10 refactoring branches later and it's a bit more sensible.
[13:43] <wgrant> with dbuser: does commit/switch/yield/commit/switch, which is usually what you want.
[13:43] <cjwatson> Yup.
[13:43] <cjwatson> Or with lp_dbuser.
[14:02] <deryck> Morning, all.
[15:16] <bigjools> howdy kirkland
[15:16] <kirkland> bigjools: howdy :-)
[15:16] <bigjools> practising my Texan
[15:16] <kirkland> bigjools: okay, so we have a commercial LP license
[15:16] <bigjools> anyway! I think it's fine
[15:16] <kirkland> bigjools: well done :-)  You wear the jersey, already :-)
[15:17] <bigjools> provided that it's done only for private PPAs
[15:17] <bigjools> yes I've been indoctrinated with Burnt Orange
[15:17] <kirkland> bigjools: heh, well, i'm all aggie, but i won't hold that longhorn affliction against you ;-)
[15:17] <bigjools> also it should be opt in on the +edit page and only enabled if the PPA is private
[15:18] <bigjools> however there are some repercussions of doing this
[15:18] <kirkland> bigjools: okay, so a "publish source", as an option, default to true for legacy compatibility
[15:18] <bigjools> aggie? I've been told that you guys shag sheep!
[15:18] <bigjools> vicious rumours I'm sure
[15:19] <kirkland> bigjools: :-P
[15:19] <bigjools> moving on ...
[15:19] <bigjools> if we don't publish sources, they will remain as PENDING forever
[15:19] <bigjools> which slows down the publisher
[15:20] <kirkland> bigjools: can we publish them and then purge them immediately?
[15:20] <bigjools> so I will need to look at a way around that
[15:20] <bigjools> well the other option is to mark them PUBLISHED but not actually publish
[15:22] <kirkland> bigjools: okay
[15:22] <bigjools> kirkland: can you file a bug about this so we can discuss there and I can subscribe wgrant who will no doubt have a strong opinion
[15:22] <kirkland> bigjools: any idea of how hard or complex this would be?
[15:22] <kirkland> bigjools: sure, against launchpad itself?  or another component?
[15:22] <bigjools> not particularly hard at all for someone with LP experience
[15:22] <bigjools> yes lp
[15:23] <bigjools> you need a schema change, a model change, view code changes and a publisher change
[15:24] <bigjools> it's the latter that needs careful attention
[15:25] <bigjools> for example, if we mark them PUBLISHED and they are not and you change your mind with the flag, then you can never publish those sources
[15:26] <kirkland> bigjools: https://bugs.launchpad.net/launchpad/+bug/919241
[15:26] <_mup_> Bug #919241: provide an option to no publish sources in a private ppa <Launchpad itself:New> < https://launchpad.net/bugs/919241 >
[15:26] <bigjools> there's currently a method that fetches all PPAs with pending publications, and it does that by UNIONing two queries, one for sources one for PPAs.  It could be split into two methods and optionally unioned by the main publisher if sources are needed
[15:26] <bigjools> kirkland: I'll comment on there with my thoughts.
[15:27] <kirkland> bigjools: perfect, thanks
[15:27] <bigjools> fixed the bug title :)
[15:27] <kirkland> bigjools: okay, perhaps this is a bit over my head, then
[15:28] <bigjools> no really, it's not too bad
[15:28] <bigjools> as far as LP patches go :)
[15:28] <bigjools> I'll describe a proposed change
[15:28] <kirkland> bigjools: rockin'
[15:29] <kirkland> bigjools: I've subscribed two of my colleagues (who are also in #launchpad)
[15:29] <kirkland> bigjools: its their work that is currently blocking on this bug
[15:29] <bigjools> cool
[15:29] <kirkland> bigjools: we're doing some really ugly work arounds in the mean time
[15:29] <kirkland> bigjools: and it's just not pretty
[15:29] <bigjools> I bet
[15:32] <kirkland> bigjools: last thread of questions...  walk me through the dev process for getting something like this implemented
[15:32] <bigjools> one sec
[15:33] <kirkland> bigjools: does it need to land in a blueprint or a sprint or work its way through a backlog?
[15:36] <bigjools> kirkland: no, none of that is needed.  JFDI.
[15:36] <kirkland> bigjools: cool
[15:36] <bigjools> someone will be happy to mentor you although I am not sure how much time I will personally have
[15:36] <kirkland> bigjools: okay, any pointers to suitable mentors for this one?
[15:37] <bigjools> so anyone can guide you through the landing process, but you need a soyuz expert to review the changes
[15:37] <kirkland> bigjools: can you give me a few l33t soyuz nicks?
[15:37] <bigjools> which is basically me, wgrant, StevenK and jtv to some extent as he has worked on the publisher. He might kill me for saying that though :)
[15:37] <kirkland> bigjools: :-)
[15:38] <jtv> bigjools: this is where the "hey let's get together sometime in the coming weeks" becomes useful.
[15:38] <bigjools> once my question in the bug is answered, this is a really easy change
[15:42] <jcsackett> Laney: did you see your branch test results?
[15:44] <rick_h> jcsackett: yea, she fixed it and I'm rerunning the tests for her
[15:45] <rick_h> jcsackett: hope you don't mind, I responded to the loggerhead reviews and attached you for follow up review since we talked/started looking at those yesterday
[15:45] <jcsackett> rick_h: don't mind at all.
[16:15] <rick_h> deryck: in checking the diff of devel vs use-convoy I'm seeing a bunch of python -> python2.6 hard coded?
[16:17] <rick_h> hmm, must be generated because I don't see it in the MP diff
[16:31] <deryck> rick_h, that's a buildout thing. I did wonder if python version mattered, based on my quick look at the error
[16:35] <rick_h> deryck: yea, sending an email back with my findings so far
[16:35] <rick_h> in case it jogs anything in your head
[16:35] <rick_h> but forgot to add the -dev list
[16:53] <sinzui> deryck, do you have a few minutes to discuss some incomplete widget javascript
[16:55] <deryck> sinzui, sure.
[16:55] <deryck> hangout or mumble?
[16:55] <sinzui> I will try hangout
[16:56] <deryck> k
[16:57] <sinzui> Looks like I can start a hangout from my phone if I send a message to you
[16:57] <deryck> ah ok, I started one and invited you.
[16:57] <deryck> I'll kill mine.
[17:00] <deryck> sinzui, you don't show up for me.  I don't mind jumping to mumble instead.
[17:01] <sinzui> you start and I will join
[17:01] <sinzui> I am still learning this
[17:04] <sinzui> deryck, https://bugs.qastaging.launchpad.net/launchpad/+bug/583392
[17:04] <_mup_> Bug #583392: IntegrityError raised setting a branch for a project series. <branches> <easy> <lp-code> <oops> <series> <Launchpad itself:Triaged> < https://launchpad.net/bugs/583392 >
[19:46] <wgrant> kirkland: It's far harder than it seems.
[19:46] <wgrant> kirkland: Because private PPA builds get the source files from the published archive.
[19:46] <wgrant> kirkland: I suspect what you really want is to be able to copy binaries without sources into another PPA
[19:46] <kirkland> wgrant: okay
[19:46] <wgrant> But that is a terrifying proposition.
[19:47] <kirkland> wgrant: *that* would be phenomenal
[19:47] <kirkland> wgrant: we're happy with our builds
[19:47] <kirkland> wgrant: but we *only* want to publish the debs
[19:47] <kirkland> wgrant: for some projects
[19:47] <kirkland> wgrant: why is that a terrifying proposition?
[19:48] <kirkland> bearing in mind that i hate non-free software as much as the next guy
[19:48] <kirkland> (the next guy in this channel)
[19:48] <wgrant> kirkland: I was ignoring the non-free bit. It's terrifying because pretty much all of Soyuz is based around being able to have the sources there as well.
[19:49] <kirkland> wgrant: urgh
[19:49] <wgrant> I'm not quite sure how to model this.
[19:49] <wgrant> It's certainly doable.
[19:49] <wgrant> And the basics would be easy to get working.
[19:49] <kirkland> wgrant: maybe we need to just run our own archive.gazzang.com and just mirror the debs from the private ppa
[19:49] <wgrant> But lots of stuff would probably break down the line.
[20:07] <kirkland> wgrant: bummer
[21:32] <abentley> bac: Could you please review https://code.launchpad.net/~abentley/launchpad/bugcomment-as-icomment/+merge/89497 ?
[21:33] <bac> abentley: sure
[21:33] <abentley> bac: Thanks.
[21:46] <bac> abentley: nice.  very clear branch.
[21:46] <abentley> bac: Thanks.
[22:23] <sinzui> bac: are you still reviewing? do you have time for https://code.launchpad.net/~sinzui/launchpad/dsp-vocab-fixes/+merge/89505
[22:23] <bac> sinzui: sure
[22:28] <bac> sinzui: done
[22:29] <sinzui> thank you bac.
[22:29]  * sinzui spell checks test module