[00:13] FFS [00:13] devel build failed [00:18] thumper: Doesn't he probably not want a shared repo? [00:18] Sounds like he might just want a branch. [00:18] * thumper nods [00:38] w00t! [00:38] don't know who fixed the swift mailing list situation ... but thanks [00:38] sinzui: was that you? [00:39] mtaylor, me and Chex [00:39] I am waiting on my test email to complete [00:39] sinzui: well thank you to both of you [00:39] mtaylor: spm did a lion share of work previously, too. [00:40] * mtaylor hands Chex, sinzui and spm beers [00:40] mtaylor: heh, glad it's sorted. [00:41] mtaylor: glad to help, sinzui did a lot of the work today, I just saw the outstanding answer and jumped in to move it along [00:41] spm: now we've just got that can't see launchpad.net/swift thing left, but I think bac may have figured that one out and filed a bug [00:41] mtaylor: that's not because I had a hissy fit of frustration the other day and dropped those tables from the DB? [00:41] it's almost like I'm going to have nothing more to bug anyone aboud [00:41] about [00:42] spm: hehe [00:42] I have not gotten a copy of my test email, and I do not see the archive [00:42] mtaylor, do you see a moderation link on https://edge.launchpad.net/~swift to verify my missing message is stuck? [00:45] sinzui: I see a moderation link, but I see no messages in the queue [00:46] :( [00:47] I think the new list was not fully created because of the stalled proc and our deft axe-like hacks to make this work [00:47] I propose a rapid deactivate, purge, and request of a new list. I think this will take a total of 10 minutes [00:53] sinzui: do you need me to do that? or was that a back-end operation you're suggesting? [00:54] I just did it all [00:54] I am sending another test [00:54] I see an archive before I even start this time :) [00:55] Sweet [00:56] mtaylor. I think you have a list. And I just learned the minimum total tear down and setup of a list in the ui is 3 minutes [00:56] sinzui: w00t! [00:57] * thumper lunchinates [01:00] thumper, spm, do you want a hand with bug 610297 [01:00] <_mup_> Bug #610297: stable codeline is failing to build - compile error [01:01] ah apparently not [01:01] :-) [01:02] 'morning poolie, spm [01:02] poolie: Did you have a good flight back to Sydney? [01:02] hey jelmer! how goes! [01:03] Very well, thanks! :-) [01:04] hi jelmer, it was ok [02:29] bdmurray: https://bugs.edge.launchpad.net/malone/+bug/611115 [02:29] <_mup_> Bug #611115: timeout: bug notifications are calculated in-request === Ursinha is now known as Ursinha-afk [03:00] mtaylor: the fix is in [03:09] bac: that's a really easily misconstrued statement. just sayin'. ;-) [03:10] spm: really? [03:10] spm: you employing some aussie linguistic jujitsu? [03:10] only for australians [03:11] bac: hrm. possibly local slang? 'the fix is in' is another way of saying 'we've done some illegal stuff to achieve X' [03:11] spm: its aussie vernacular [03:11] spm: ok. well rest assured it was all legal, reviewed, and landing soon [03:11] heh [03:13] and look, you ran off wgrant. launchpad productivity plummets [03:14] on the grounds that he's only 7-8 hours drive away; i make no comment in case he's the revenge seeking type [03:14] e.g. australian ? [03:15] :P [03:15] .... [03:16] ha. good def'n here of 'the fix is in' http://answers.google.com/answers/threadview?id=65441 fwiw [03:18] ah right [03:18] so less aussie than I thought [03:20] bac: That was actually only one of me. [03:20] (my server at home apparently reset itself just as I was SSHing in... this is concerning) === wgrant_ is now known as wgrnat === wgrnat is now known as wgrant [03:36] Is devel broken at the moment? [03:36] I have a test failure which seems to be caused by the registration slot changes. [03:39] wgrant: it is possible [03:39] wgrant: there have been many commits since the last successful run [03:39] Eep. [03:39] But the registration slot is on edge... [03:44] this is odd [03:47] xx-copy-packages.txt is the failure that I see. [04:30] wgrant: should the fact that there are 13 idle i386 builders be bothering me? [04:31] ppa builders, that is [04:34] Maybe. [04:34] If it's for more than a few minutes. [04:35] * wgrant watches. [04:35] ah a bunch just got dispatched [04:35] You just caught it at a bad time, yeah. [04:36] so it was probably just process-upload-ing openoffice or something [04:36] Heh. [04:36] Each p-u invocation takes like 15 seconds. [04:44] So if it is processing a couple of dozen uploads, it can take a while to get around to the next cycle. [04:44] Even if those uploads are small. [05:02] rockstar: around? [05:03] so - now I'm seeing work-in-progress merge requests on the +activereviews page ... [05:04] oh. even weirder [05:04] +activereviews for the branch shows work-in-progress - +activereviews for the project does _not_ show work in progress [05:04] just thought I'd point that out in case no body noticed [05:46] lifeless: quick question [05:46] shoot [05:46] mtaylor: yes it does [05:46] mtaylor: by design [05:46] lifeless: the librarian bug you're fixing, will this stop the timeouts we are seeing on merge proposal pages? [05:47] mtaylor: work in progress is of interest in the personal namespace [05:47] maybe. Point me at such a timeout OOPS [05:47] mtaylor: branches are in the personal namespace [05:47] lifeless: they are in the daily oops reports [05:47] * thumper has a quick look [05:47] lifeless: OOPS-1670O1242 [05:48] * thumper looks at _mup_ [05:48] works in #launchpad :P [05:48] yes, I would expect that one to be fixed [05:48] we'll allocate a token [05:49] \o/ [05:49] and the client will grab the content directly from the librarian [05:49] um... [05:49] that'll require a coding change won't it? [05:49] it isn't a transparent fix [05:50] because we format the diff [05:50] the formatted diff isn't stored [05:50] ok [05:50] so no, that one won't be fixed. [05:50] :( [05:50] * thumper afk [05:50] Heres what will be fixed - places where we just proxy the content to the client. [05:50] like private build logs [05:53] thumper: the separate 'client does not time out fast' bug will help with those pages though - but it will just mean they fail fast rather than fail slow. [05:55] lifeless: Hum, so the restricted librarian is going to visible externally? [05:56] +be [05:56] wgrant: not precisely [05:57] wgrant: we'll accept requests on the public librarian for secured content if there is a Front-end-https: on header and it has an access token that is live in the session db [05:57] wgrant: see my WIP branch / bug discussion [05:57] Ah, sounds reasonable. [05:57] Well. [05:57] Sort of reasonable. [05:57] As long as you use the right Content-Disposition... [05:57] can you suggeset some improvements ? [05:58] How is c-d different for this case than other librarian objects? [05:58] The public librarian doesn't use Content-Disposition: attachment. [05:58] So JS is executed. [05:59] right [05:59] So if you serve anything from the HTTPS librarian without Content-Disposition: attachment, you're fucked, because anyone can steal your access tokens. [05:59] how is this *different* in terms of requirements/risks/constraints ? [05:59] no [05:59] the token is per url [06:01] we can't use the oauth tokens for the reason you give [06:01] Relying on cross-window security seems a bit odd. [06:01] even if we set c-d they might exploit a browser bug etc [06:01] They might, yes, but then the entire Internet is fucked. Not just us. [06:02] wgrant: so still, that seems like an argument for setting it on every librarian file [06:02] how is it unique for secured content [06:02] lifeless: Because there's nothing to be stolen on the public librarian. [06:03] That's why it lives on launchpadlibrarian.net nowadays: so there's no cookie. [06:03] the secured still will also live there [06:03] and also have no cookie [06:03] But there is private information. [06:03] In the URLs. [06:03] go on [06:04] I'm moderately fucked from jetlag, so you're going to have to treat me like a 2-year old and spell it out :) [06:04] I'm aware of strong cross-domain security rules. [06:04] I don't know about cross-window restrictions, though. [06:05] what do you mean cross-window restrictions [06:06] The security of your system relies on the fact that a page executing in the context of a domain cannot determine the URLs of any other windows open for that domain. [06:06] I do not know that to be the case. [06:07] or of urls in the history [06:07] That too. [06:09] so if you're saying 'please set cd:attachment on secured urls' - sure, I'm fine doing that, I don't see that we would want js scripts in the librarian anyhow for secured content [06:09] but it seems to me that we're already at risk from such things - browser history containing product urls etc which are private [06:09] Nobody else has managed to do that without being harrassed for days. [06:10] please expand [06:10] U1 was pretty badly broken in this respect. [06:10] Took a bit of poking to get it fixed. [06:10] But I don't see your point about private URLs. [06:10] I don't know what 'that' or 'this' is bound to. [06:11] The serving user web pages on authenticated domains thing. [06:11] The only untrusted content that LP currently serves up is on ppa.launchpad.net and launchpadlibrarian.net. [06:11] ppa.launchpad.net is HTTP, so can't access cookies. [06:12] launchpadlibrarian.net is on another domain, and contains no private content, so it can't access cookies or private content. [06:12] right [06:13] now I'm proposing to add secured content to ll.net by using the url to carry the token [06:13] it can't access cookies - wrong domain [06:14] But it's not clear if it can access other windows or history. [06:14] right [06:15] if we add c-d to the headers, will that break things like emblems, if they are secured ? [06:16] thumper: interesting [06:16] lifeless: Not sure. [06:17] wgrant: so, I'm not clear if you're saying 'stop this the plan is flawed', or 'the plan needs tweaking', or 'this axiom needs researching' [06:17] wgrant: perhaps you're being clear and I'm just too freakishly tired [06:17] lifeless: Probably "this axiom needs researching" [06:18] wgrant: ok, cool - I can keep plodding forward. [06:18] wgrant: if you wanted to research a little, to parallelise efforts, that would be rad. [06:18] wgrant: or, if there is a good mitigation strategy we can use regardless, lets do that. [06:18] lifeless: Content-Disposition is the way to do it, I believe. [06:19] we could, in principle, wire dns up to the librarian [06:19] and do hash.launchpadlibrarian.net [06:19] but that seems a little overkill [06:22] wgrant: do you think it would be safe to whitelist file types like gif, svg, txt etc ? [06:23] lifeless: IE does some pretty strange stuff. So you'd have to be careful. [06:24] so the vector you're describing is: [06:24] Attacker puts a bad file in the librarian [06:25] Victim looks at that file, and has either in that windows history, or in another window, the url for a secured file on the librarian [06:25] Attackers script downloads the secured file (because its within the timeout/use count for the token) and sends it off (how?) to another site [06:26] It's more practical to just transmit the URL. [06:26] Either through redirection, or a form post. [06:27] so they set self.url = badsite.net/?location=secured_content_url [06:27] window.location, but yes, exactly. [06:27] blah, self.location = ... [06:27] righto [06:27] one thing we can do [06:28] is look for a referrer which must exactly match the canonical url in LP [06:28] [note that this is hard today because we don't have a backlink on LFA, I'm speculating] [06:28] gesturing... handwaving, thats what I mean, handwaving. [06:28] If you add more referrer sniffing, I will do bad things. [06:29] heh [06:29] (yes, I did suggest it for the CSRF fix, but only because I knew the hole had no chance of being fixed within years otherwise) [06:29] now, the referrer may be no more secure [06:29] I'd hoped that the user complaints would force Foundations to fix it properly. [06:29] But I was wrong. [06:29] so that might be pointless [06:30] c-d will break inline display of diffs, logs, etc - which is annoying. [06:30] security over pretty though [06:31] Right. [06:31] theres been some content-sniffed spec stuff recently [06:32] probably want to review that [06:36] Hmm. [06:36] I wish things like getUtility(IComponentSet)['main'] would cache. [06:36] ugh [06:36] Hm? [06:36] I wish such things didn't really exist. Or something. [06:37] Well, yes. [06:37] I'm currently trying to refactor some publication stuff. [06:37] But I can't do it really cleanly, since there were apparently timeouts caused by too many getUtility(IComponentSet)['main'] calls. [06:38] heh [06:38] I hear objects provide a nice context for remembering things [06:38] I guess we could just assume it never needs to be invalidated. [06:39] Ugly and wrong, but it would never be a problem ever. [06:40] And then my code can be all nice and clean. [06:40] lifeless: Am I allowed to do that? [06:40] do what [06:42] Make it cache lookups, even though someone could in theory delete and recreate the component. [06:42] within a transaction, yes. [06:42] as long as a delete & recreate in the transaction will be handled [06:43] Is there a way to do that? [06:43] have you considered providing a stateful proxy in your code [06:43] wgrant: you can hook into certain events in storm, but I think it would be ugly. [06:44] It would be very ugly, yes. [06:44] Especially for a situation that will never occur. [06:44] never is a long time [06:44] have you considered providing a stateful proxy in your code? [06:45] How do you mean? [06:46] class mycomponentset: def get_component(self, component_name): if component_name not in self.components: self.components[component_name] = getUtility(IComponentSet)[component_name]; reuturn self.components[component_name] [06:46] I was hoping to be able to do that in ComponentSet itself. [06:47] the context for such a cache is ill defined at the component level [06:47] AIUI [06:48] Well. Components are currently immutable. [06:48] Like SourcePackageNames and BinaryPackageNames. [06:48] They are just a string. [06:48] update on the name is prevented ? [06:48] No. But it's never done, because it would break the world. [06:48] as long as you really clearly document it then [06:49] And while it's *possible*, it seems bad to make the code significantly uglier to take into account that case. [06:49] be sure not to leak objects across transactions [06:49] you'll want to hook into end request to clear your cache [06:49] Yeah, that's the bit I'm not quite sure about. [06:49] and if you're used outside web contexts you'll need to provide an API for clearing the cache and audit existing code to comply [06:50] global caches are a pain [06:50] for an example of hooking into end request see profile or feature flags [06:52] I'm slightly surprised there isn't some sort of @cacheduringtransaction decorator. [06:52] It would be handy. [06:52] I suppose the tricky bit is telling the decorator which store to use. [06:54] (There is @cachedproperty, but it caches indefinitely, although you could invalidate it by hand fairly easily...) [06:54] Right, but it doesn't help for attribute accesses. [06:55] Ah. [06:55] I could use a celebrity. [06:55] really? [06:55] That sounds evil, but they have the rules that I want. [06:55] That's true. [06:55] cause I need a hero [06:57] lifeless: Why? When will you return the last one you borrowed? [06:58] :) [07:21] wgrant: http://code.google.com/p/browsersec/wiki/Part2#Same-origin_policy - 'The behavior for URLs that do not meaningfully have a host name associated with them (e.g., file://) is not defined, causing some browsers to permit locally saved files to access every document on the disk or on the web;he behavior for URLs that do not meaningfully have a host name associated with them (e.g., file://) is not defined, causing some browsers [07:21] bwah c&p fail [07:29] lifeless: is np. simply went intot the tl;dr bucket anyway ;-) [07:41] lifeless: that's a fairly depressing document [07:41] yes [07:44] A long-standing security flaw in Microsoft Internet Explorer 6 permits stray newline characters to appear in some XMLHttpRequest fields, permitting arbitrary headers (such as Host) to be injected into outgoing requests. This behavior needs to be accounted for in any scenarios where a considerable population of legacy MSIE6 users is expected. [07:44] aaaaaaaaaaa [07:53] wgrant: see 'Gaps in DOM access control' on that page [07:55] wgrant: so interestingly, *any* page can get any url [07:56] wgrant: for many browsers; so urls that we consider private, like private team names, are obtainable (though requires both guessing and a named window AFAICT) [08:08] " where a considerable population of legacy MSIE6 users is expected. " ... that give me nightmares [08:41] mtaylor: That sadly accounts for a significant portion of Launchpad's privacy userbase, I believe. [08:44] rumoured, not confirmed [08:47] good morning [09:12] Hello [09:25] hello europa [09:26] thats a whole new tz :) [09:30] you still here? [09:30] what's the cleanest way to get my docs into something visible on the web? [09:30] put them in a module docstring? [09:30] Probably. [09:39] yeah, for now, docstrings [10:06] Can someone else run xx-copy-packages.txt? It fails for me in devel. [10:06] But stable is up to date. [10:06] Even though the topic says buildbot is broken. [10:06] I am confuse. [10:15] it failed for me too [10:15] meant to mention that 2 days ago [10:16] So... buildbot is fail-deadly? === jml changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 2 of 10.08 | PQM is open | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes [10:37] jml: can we keep the bb summary please, so its obvious where to put updates to it? === jml changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 2 of 10.08 | PQM is open | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes [11:12] bigjools: Hi. [11:12] hello [11:13] bigjools: I'm currently working on DDEB copies. The binary copy logic is currently in PublishingSet.copyBinariesTo rather than BPPH.copyTo, originally for two performance improvements. [11:13] One of those was to retrieve the insecure records all in one go. That's clearly no longer an issue. [11:14] The other is to only issue one query for the PPA component override (since getUtility(IComponentSet)['main'] doesn't cache). [11:14] It seems like that second optimisation could be implemented a lot better by making ComponentSet cache responses. [11:14] yeah I agree [11:15] components simply don't change much [11:15] Exactly. They *could*, but they never will. [11:15] for Ubuntu indeed, but when we start hosting more distros, they might [11:16] Well, in that case some will be added. [11:16] But existing ones will never change or be removed... [11:16] yep [11:16] So it seems reasonable to have ComponentSet just cache positive responses within a transaction. [11:18] +1 [11:18] Great. [11:19] Not sure how best to go about it, though... unless I go the celebrity route and store the ID at first request, and then rely on Storm's cache. [11:19] I'm not sure there are any other utilities that do a similar thing. [12:04] jtv: What happened to the work to drop BranchRevision.id ? [12:05] stub: John went on with that... I pushed him some to keep going. [12:05] We landed the stormification of the queries. [12:06] Cool. I'm getting concerned meeps from the losas about disk space, and that should reclaim maybe 15GB [12:06] psiew [12:06] (that's a sound, not an acronym) [12:07] * wgrant blinks. [12:07] 15GB from removing the integer primary key of that table? [12:08] jam: see above... getting rid of BranchRevision.id is increasingly urgent. [12:09] The index itself is about 12.9GB, plus id storage, minus bloat = [12:09] Ah. [12:09] stub: if this is an append-only table as it seems to be, I'm not expecting much bloat that vacuums wouldn't keep under control. [12:10] yer, there isn't [12:11] The id storage is presumably 4GB for every G rows. There's also another index that contains id, so add that. Yeah, this would be a nice cleanup. [12:22] stub: I'd say the savings would be closer to 20GB. [12:41] today is not working out well for me [12:41] I may have to edit Python code to soothe my frazzled nerves. [13:00] Hi, all. [13:03] 'morning Deryck === matsubara-afk is now known as matsubara === salgado is now known as salgado-dr [13:29] lifeless, ping [13:38] gary_poster, hello [13:39] gary_poster, I have an idea lurking in the back of my mind to move some of the more strategic docs that I'm writing to the Launchpad tree, as well as some of the developer documentation [13:40] gary_poster, but I think I'd only want to do that if the docs were also browseable on the web [13:41] gary_poster, technology-wise, should I be using the apidoc book for this, or some kind of sphinx thing? === Ursinha-afk is now known as Ursinha [14:30] jml, I'll summarize pros and cons in 15 [14:31] gary_poster, thanks === Ursinha is now known as Ursinha-afk [14:51] hi jml [14:51] lifeless likely won't be on for another 4-6hours === Ursinha-afk is now known as Ursinha [14:55] jml: Sphinx: great if you need links between pages or more sophisticated rendering possibilities--closer to a website or a book. apidoc: great if you need quick ReST processing that's integrated with something we already have, and can be loosely organized into a tree. [14:55] downside of both: getting them on the web is a manual process, and likely to stay that way at least for awhile. (I was vaguely imagining a buildbot/hudson thing that would do this eventually for the apidoc.) It's very slow for the computer to generate the static apidoc and upload it, but no big deal for the person pushing the keys. [14:55] For the kind of docs I *expect* you'll be writing, I'd lean towards apidoc, especially if I can shunt off the extraneous stuff (like ZODB) so the docs valuable to us are easier to see [14:55] jam: thanks. [14:55] adeuring, hi. So I'm trying to catch up here on the proxied (or not) factory objects discussion. Do we have resolution on that? i.e. are you still working on it, or has the work been abandoned now? [14:56] I need to wade through that discussion :-/ [14:56] gary_poster, thank you, that's helpful. [14:57] deryck: i've abandoned it for now. it is easy to pick up once we know how to proceed with (un)proxied objects from the factroy [14:57] cool np [14:57] gary_poster, although you missed that sphinx looks prettier than apidoc :) [14:57] jml: very true :-) [14:57] adeuring, is bdmurray unblocked on the work he wants to do? Or has a work around at least for the test problem he had? [14:57] deryck: he's making some progress. sorry forgot the details [14:58] adeuring, so the card for you and bug 610000 is abandoned? [14:58] <_mup_> Bug #610000: add security proxies to the objects returned by LaunchpadObjectFactory.makeMergeDirective() and LaunchpadObjectFactory.makeMilestone() [14:58] right [14:58] adeuring, can you move it to trash on the board, and add a comment why you're abandoning the work? [14:59] ok, will do [15:02] deryck: do we have a trash lane on the kanban board? [15:03] adeuring, yeah, if you right click the card, move to --> archive --> trash. You'll need to then expand archive to find it and comment, or else comment before you move it. [15:04] deryck: thanks, found it now [15:06] cool [15:06] *sigh* can't even spell my own name today === salgado-dr is now known as salgado [16:12] gary_poster: fwiw, I've got a hudson job that build sphinx doc for our openstack projects and publishes them to web on every trunk push [16:13] mtaylor: sweet, sounds nice :-) [16:16] gary_poster: yes. I gives me pleasure [16:16] it [16:16] not I [16:16] gah [16:16] lol === beuno is now known as beuno-lunch [16:35] gary_poster, hi. Do you have 5 minutes to chat? [16:36] deryck, on call, and booked. in hour, maybe? [16:37] gary_poster, sure. sounds good. No hurry. [16:37] Just trying to figure out why my changes last week caused test failures and had to be reverted, and I think you might have some idea. :) [16:37] heh ok :-) === beuno-lunch is now known as beuno === matsubara is now known as matsubara-lunch === Ursinha is now known as Ursinha-lunch === deryck is now known as deryck[lunch] [18:09] I've landed a branch on edge to cache a bug target's bug tags portlet but I'm not seeing cache comment in the source for the page [18:13] gary_poster: is there somebody who could help me sort this out? === Ursinha-lunch is now known as Ursinha [18:24] bdmurray, are you sure it's been rolled out? [18:25] jml: pretty sure - it is revision number 11195 [18:25] speaking of it'd be neat if 'r11249' linked to the code branch [18:25] that does give you a certain ground for confidence [18:26] bdmurray, yeah, that would be neat. [18:27] * jml hacks a little [18:28] bigjools, ping [18:28] rockstar: is it quick, I'm about to EOD? === matsubara-lunch is now known as matsubara [18:30] bigjools, just wondering if there's a factory method for simulating the creation of all objects that would be created to have a package in a PPA. [18:30] rockstar: yes, the SoyuzTestPublisher [18:30] bigjools, okay, I'll take a look at that. [18:30] bigjools, have a good night. [18:30] bdmurray, reckon it should link to http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/files/11249 ? [18:31] rockstar: there's a getPubSource() which creates a package and publishes it [18:31] rockstar: there's a few gotchas with it, see some example tests first (grep is your fwiend) [18:31] bdmurray: code only adds comments "if not config.launchpad.is_lpnet and not config.launchpad.is_edge:" [18:31] bigjools, okay. I need to allow that to grow the ability of using a sourcepackagerecipe to do so. [18:31] rockstar: sounds great [18:32] bdmurray: so it would be visible in staging...if staging were up. [18:32]