[00:49] <mars> lifeless, do you know of anyone who has a .testr.conf for nose?
[00:49] <mars> lifeless, I know about the TestID plugin for nose, but using testr itself would be nice
[01:13] <mars> lifeless, turns out to be pretty simple: http://pastebin.ubuntu.com/488515/.  But for some reason it will not pick up the list of failing tests.
[01:14] <mars> ah ha!
[01:18] <mars> lifeless, you need to redirect STDERR for it to work with nose.  This /almost/ works as expected: http://pastebin.ubuntu.com/488516/
[01:57] <lifeless> mars: sweet
[01:58] <lifeless> no idea why it would be writing subunit to stderr.
[01:58] <lifeless> still, mine not to wonder why
[05:55] <lifeless> I've no idea how its getting negatives
[05:57] <lifeless> finish sets duration to a timedelta of 'NOW - start'
[05:57] <lifeless> start is set to NOW
[05:57] <lifeless> NOW - NOW = 0, so the duration should be the change in NOW between the two calls being made.
[05:57] <lifeless> jtv: &
[05:57] <lifeless> jtv: ^
[05:58] <lifeless> I wonder if its an errorlog bad-responsibilities thing
[05:59] <jtv> lifeless: you're sure only one clock is involved?
[05:59] <lifeless> yes
[05:59] <lifeless> the durations go down almost linearly as the offset increases
[05:59] <mwhudson> context?
[05:59] <lifeless> so something is substracting duration again
[06:00] <lifeless> bug 630612
[06:00] <_mup_> Bug #630612: Complete b0rkage of oops timing info <Launchpad itself:New> <https://launchpad.net/bugs/630612>
[06:00] <lifeless> e.g. https://lp-oops.canonical.com/oops.py/?oopsid=1708EB636
[06:00] <jtv> Hmm it's definitely not clock skew.
[06:00] <lifeless> add offset to the times and its right
[06:00] <lifeless> I think
[06:01] <jtv> Are you accidentally throwing the baseline time into the subtraction?
[06:01] <lifeless> no
[06:01] <lifeless> definitional issue
[06:01] <lifeless> putting up a patch
[06:01] <jtv> As in, finish - start - baseline?
[06:03] <lifeless> no, its that it doesn't want a duration
[06:03] <lifeless> the old code wasn't tested
[06:03] <lifeless> and I misunderstood what the disk format was meant to represent
[06:07] <lifeless> https://code.edge.launchpad.net/~lifeless/launchpad/oops/+merge/34635
[06:08] <lifeless> thumper: are you around, perchance ?
[06:08] <thumper> not really
[06:08] <thumper> whats up?
[06:09] <lifeless> I broke oopses on edge; fix is trivial, PQM is closed, needs a release-critical stamp
[06:09] <lifeless> which is you, or gmb in about 3-4 hours
[06:09] <thumper> lifeless: that mp up?
[06:09] <lifeless> https://code.edge.launchpad.net/~lifeless/launchpad/oops/+merge/34635
[06:09] <lifeless> by broke, they work, but you need to think like a pretzel to analyse them
[06:09] <thumper> diff not there yet
[06:10]  * lifeless bets its hung
[06:10] <lifeless> I've pasted a diff in
[06:11] <thumper> lifeless: the diff and comment don't really help me understand it
[06:11] <thumper> but if you are sure it is right
[06:11] <thumper> I'll rc it
[06:12] <lifeless> thumper: I can explain it pretty quickly
[06:12] <thumper> go then
[06:12] <lifeless> look at this oops
[06:12] <lifeless> https://lp-oops.canonical.com/oops.py/?oopsid=1708EB636
[06:12]  * thumper is supposed to be making dinner
[06:12] <lifeless> specifically the sql log
[06:12] <lifeless> notice how the durations go negative
[06:12] <lifeless> the actual duration is the reported duration + the start offset.
[06:12] <thumper> aye
[06:13] <lifeless> my patch adds the start offset in the generation code
[06:13] <thumper> ok
[06:13] <lifeless> I misunderstood what the disk format was meant to have in it.
[06:13] <thumper> done
[06:13] <lifeless> thanks
[06:13] <lifeless> jtv: care to provide the code review stamp that will be wanted by ec2land ?
[06:14] <jtv> lifeless: since you're phrasing it like I'll have a meaningful role in the process, sure
[06:15] <lifeless> This won't hit sundays rollout, but I can nurse it into devel tonight and ask spm to trigger a reroll of edge monday am
[06:15] <lifeless> sorry about breaking it
[06:21] <lifeless> jtv: I think you do have a meaningful role - there are three people that have read the entire change here: you, me, mwhudson
[06:21] <lifeless> jtv: I can't think of anyone better suited to review this (trivial) patch
[06:23] <jtv> Well, it's reviewed.
[06:24] <lifeless> thank you
[06:24] <jtv> The only way I could find to do that was to request a review from myself.
[06:24] <lifeless> oh
[06:24] <lifeless> all you have to do is type in the comment box
[06:24] <jtv> I could've added an Approve vote, but not with a specific review type.
[06:24] <lifeless> and change the drop-down to approve/needs info etc
[06:24] <jtv> And you want the "code" review type for "ec2 land."
[06:24] <lifeless> ah
[06:24] <lifeless> I think a default type == code, doesn't it ?
[06:27] <lifeless> jtv: thanks for catching this
[06:27] <lifeless> I would have when I got a chance to look, but I'm glad to catch it as early as possible
[06:44] <lifeless> I so want a button I can push.
[06:44] <lifeless> which will deploy.
[06:47]  * mwhudson is tempted to say that lifeless almost certainly owns a device which can cause a deployment by pressing lots of buttons in the right order
[06:49] <mwhudson> lifeless: i've just read set_request_timeline :)
[06:49] <mwhudson> s/:)/:(/
[06:49] <lifeless> mwhudson: I would deeply deeply love to have that better..
[06:50] <lifeless> mwhudson: I'm going to be putting it fairly high up in my hygiene requests for foundations I think; can't build on a shaky base.
[06:52] <lifeless> DistributionSourcePackage:+addquestion is looking pretty unhealthy
[06:54] <mwhudson> lifeless: did you do any coding towards bug 623199 ?
[06:54] <_mup_> Bug #623199: scripts do not establish valid zope partiticipations <Launchpad Foundations:New> <https://launchpad.net/bugs/623199>
[06:55] <lifeless> mwhudson: nothing reusable
[06:55] <lifeless> mwhudson: I scrapped it as a learning experience
[06:55] <mwhudson> i could have a hack now, but i guess an hour is probably extremely optimistic to get something useful done
[06:55] <mwhudson> lifeless: what did you learn?
[06:56] <lifeless> I learnt that our scripts code is horribly confused about what they do
[06:56] <mwhudson> :(
[06:56] <lifeless> I am more and more of the opinion we want impersonation
[06:56] <lifeless> we want something to be able to:
[06:56] <lifeless>  - run async
[06:57] <lifeless>  - use the API to do shit
[06:57] <lifeless>  - do it on behalf of a user that originated the work
[06:57] <lifeless> I think the second line, when written 'use SQL to do shit'
[06:57] <lifeless> should mean nearly-no-code-changes, ideally.
[06:58] <lifeless> e.g. scripts should setup a participation of the user who the work is on behalf of
[06:58] <mwhudson> well i can see that would be good
[06:58] <lifeless> this is obviously only applicable to deferred-work-scripts
[06:58] <mwhudson> but can't we make canonical.launchpad.webapp.adapter less terrible without that?
[06:58] <lifeless> others like the PPA access token updater are conceptually different
[06:58] <lifeless> mwhudson: sure
[06:59] <mwhudson> lifeless: and by run async, you mean in the context of appserver requests?
[06:59] <lifeless> no
[06:59] <mwhudson> not twisted scripts that somehow process more than one job at once?
[06:59] <lifeless> badly phrased
[06:59] <lifeless> 'run out of step with appserver stuff'
[06:59] <mwhudson> ok, that's what i thought you meant
[06:59] <lifeless> maybe for a single request, maybe much later, and all things inbetween
[06:59] <mwhudson> i phrased it badly too
[07:00] <mwhudson> :)
[07:00] <lifeless> anyhow
[07:00] <lifeless> yes, we can make adapter better
[07:00] <lifeless> I think your plan is a good one:
[07:00] <lifeless>  - a specific interface which needs the following characteristics:
[07:01] <lifeless>   - adapter and other code like it (timelines, featureflags, permission checking) can rely on [if missing the error/fail appropriate;y]
[07:01] <lifeless>  - HttpRequest implements it
[07:02] <lifeless>  - something for scripts implements it
[07:02] <lifeless> we may want the ability to push-and-pop the contextual-lookup for these objects
[07:02] <lifeless> example:
[07:03] <lifeless>  checkwatches starts up, it needs to (picking one such thing) get the timeline for it as a whole and use that while it queries the DB for watches to update
[07:03] <lifeless> it may then want to, per watch, push a new context, which will get the db queries, errors, http client times, mail sending times, for a single watch.
[07:04] <mwhudson> well
[07:04] <mwhudson> there is newInteraction and restoreInteraction
[07:04] <lifeless> sure
[07:04] <mwhudson> so zope already supports this to some extent
[07:04] <lifeless> I'm trying not to talk impl
[07:04] <mwhudson> ok
[07:04] <lifeless> you know whats available much better than I
[07:04] <mwhudson> i only learnt about restoreInteraction a couple of weeks ago :-)
[07:04] <lifeless> see
[07:05] <lifeless> you're a couple of weeks ahead of me :P
[07:05] <lifeless> thats a big fraction of my time in this job :>
[07:05] <mwhudson> well, if it took me three+ years and you a few months, that's a good sign all round i think
[07:05] <lifeless> ha!
[07:06] <mwhudson> i think launchpad suffered for a time for not having any one who really got zope
[07:06] <lifeless> I've been listening in the corners all this time
[07:06] <mwhudson> so we should make sure we hang on to gary and benji :-)
[07:06] <lifeless> mmm, we *started* with serious zopers.
[07:07] <lifeless> anyhow, we're going well now
[07:07] <lifeless> and I can see a path to having headroom to really tackle things.
[07:08] <mwhudson> yeah
[07:08] <mwhudson> that's good
[07:08] <lifeless> so webapp adapter
[07:08] <lifeless> I can - I have - described what I think we need in broad terms.
[07:09] <lifeless> building on your description on the list
[07:09] <lifeless> what it needs now is someone to implement it and migrate a couple of scripts over, such that we can see it has legs.
[07:10] <lifeless> you could probably do the start in an hour
[07:11] <lifeless> I dunno about getting into the meat
[07:11] <lifeless> time for me to put on the war of the worlds and ratchet up the private librarian refactoring
[07:13] <mwhudson> fair enough
[07:21] <mwhudson> i think i'm going to think about linaro stuff instead :)
[07:29] <lifeless> \o/ next rollout will be tracking email/librarian/memcache times
[07:29] <lifeless> badly
[07:29] <lifeless> but tracking them
[09:44] <lifeless> jtv: bug 629921 might entertain you
[09:44] <_mup_> Bug #629921: Archive:+packages with empty name search does like '%%' search. <timeout> <Soyuz:Triaged> <https://launchpad.net/bugs/629921>
[09:59] <maxb> What's the likelyhood of finding someone capable of bouncing codebrowse on a Sunday? :-/
[10:31] <lifeless> maxb: low to middling
[10:32] <lifeless> hmm, whats the recommended url parsing lib for lp code
[11:10] <wgrant> lifeless: lazr.uri, probably.
[11:11] <lifeless> wgrant: do you know of any use for https urls on librarian files?
[11:11] <lifeless> (the current system, I mean)
[11:18] <wgrant> lifeless: They're used in the webapp.
[11:18] <wgrant> To avoid insecure content warnings.
[11:18] <wgrant> eg. project icons.
[11:18] <lifeless> blah
[11:18]  * lifeless rethinks part of this
[11:19] <lifeless> the docs could at least be a little less daft about how they explain it
[11:27] <lifeless> man the layers are messy
[11:27] <lifeless> I think its been refactored and docstrings not changed.
[11:32] <gmb> lifeless, So, the current build failure is because the checked-in wadl is out of sync with what's generated by LP. I'm regenerating the on-disk wadl and checking it in; that should fix the breakage.
[11:32] <lifeless> heh
[11:33] <lifeless> so are we meant to do that always? How do we tell if a change is incompatible?
[11:33] <lifeless> could you perhaps file a bug asking these things of foundations :)
[11:33] <lifeless> actually
[11:34] <lifeless> gmb: can you please disable the test.
[11:35] <gmb> lifeless, You have a good point. I think I can shed some light on the reason for the test anyway:
[11:36] <gmb> 1. Before we had checked-in WADL, there were always bzr conflicts because people would accidentally check in the apidoc directory (which would be created if it didn't exist already)
[11:36] <gmb> 2. Therefore, it was decided to check in the WADL to prevent those conflicts.
[11:36] <gmb> 3. Trouble was that WADL generation took a long time and unless it's --forced it won't overwrite the extant files.
[11:37] <gmb> So the test is there to prevent us from having something broken rolled out.
[11:37] <gmb> lifeless, I don't think we should disable the test unless we're going to stop having the WADL checked-in.
[11:37] <lifeless> gmb: I've replied in the thread
[11:38] <lifeless> gmb: but lets check my logic.
[11:38] <lifeless> we have two branches.
[11:38] <lifeless> stable, db-devel.
[11:38] <lifeless> both receive API changes.
[11:38] <lifeless> whats going to happen in a cycle after both have had -any- API change.
[11:39] <gmb> lifeless, Your reasoning is sound.
[11:39] <gmb> lifeless, Okay, I agree; I'll disable the test.
[11:39] <lifeless> Perhaps revert the merge that added it to restore the old logic, whatever that was.
[11:39] <wgrant> Checking in WADL seems somewhat... odd.
[11:39] <lifeless> I don't know enough of the guts to suggest the right thing to do.
[11:39] <lifeless> I believe there was an additional desire to prevent API regressions by making people thing.
[11:39] <lifeless> s/thing/think/
[11:40] <lifeless> On reflection, I don't think the WADL is human readable enough for developers to do that routinely.
[11:40] <gmb> lifeless, I think that test has been around for a while, actually.
[11:40] <lifeless> gmb: really ?
[11:41] <gmb> lifeless, Yes. THough I'm not certain. I'll check now.
[11:41] <lifeless> I though benji landed it late last weke
[11:41] <lifeless> \o/ we should be getting librarian stuff in oops now. /me goes to try
[11:42] <gmb> lifeless, Oh, right. For some reason I thought it was something that had been kicking around for a while. In my head, I was blaming mars ;)
[11:42] <lifeless>  the motivations are good
[11:42] <lifeless> needs some more glue to work well
[11:44] <gmb> lifeless, You're right; it landed on devel the week before last.
[11:45] <gmb> I'll revert the merge(s).
[11:46] <lifeless> thanks
[11:46] <lifeless> my sunday is fading fast
[11:47] <gmb> lifeless, Okay, no worries. I'll take care of this.
[11:54] <lifeless> \o/
[11:55] <lifeless> something slightly screwy
[11:56] <lifeless> 15ms to connect to librarian in the dc
[11:56] <lifeless> and 0ms to get the diff down
[11:57] <lifeless> 15ms is a bit slow
[11:57] <gmb> I swear PQM keeps adding things to the regex so that my submissions fail.
[11:57] <lifeless> https://lp-oops.canonical.com/oops.py/?oopsid=1709EB904 for folk that can see
[11:57] <lifeless> gmb: what did you try
[11:57] <lifeless> and to what branch; they are funkily different
[11:58] <gmb> lifeless, Oh, I'm being facetious. For some reason it's asking for [ui=] as well as [testfix][r-c][rs].
[11:58] <lifeless> \o/
[11:58] <lifeless> what branch
[11:58] <gmb> devel
[11:58] <lifeless> freaky
[11:58] <gmb> Indeed.
[11:59] <lifeless> so this cycle
[11:59] <lifeless> when questions goes beserk, I'll be able to point and laugh at email very very easily :P
[11:59]  * lifeless bets sending email is not cheap
[12:02] <thekorn> lifeless: hi, your last comment on bug 620458 is a big surprise too me ;) the code from my last comment did not work a week ago,
[12:02] <_mup_> Bug #620458: cannot access attachments of private bugs any more <qa-needstesting> <httplib2:Unknown> <Launchpad Bugs:Fix Committed by adeuring> <https://launchpad.net/bugs/620458>
[12:03] <lifeless> thekorn: are you using production or edge?
[12:03] <thekorn> and no, I'm not running the code in your datacenter
[12:03] <thekorn> lifeless: works on both, maybe I was facing a different issue
[12:03] <lifeless> perhaps
[12:03] <lifeless> perhaps deryck and abel rolled back the privacy change
[12:03] <lifeless> I'm working on the long term fix atm
[12:03] <lifeless> hopefully we'll get it out in this rollout, and it will be faster after that.
[12:04] <thekorn> lifeless: my way to reproduce this bug was always: "attachments are not accessible for private bugreports not reported by myself"
[12:04] <lifeless> thekorn: well, I'll talk to deryck tomorrow night
[12:04] <lifeless> gmb may know stuff now.
[12:04] <lifeless> gnight y'all
[12:04] <thekorn> great
[12:04] <thekorn> good noight
[12:05] <gmb> thekorn, I'm afraid I don't know much about the private attachment stuff besides that there's still work ongoing. I'll speak to adeuring in the morning.
[12:08] <thekorn> gmb: no problem, just wanted to give lifeless a quick answer to the question he had in his last comment
[12:09] <gmb> thekorn, Ah, okay, cool.
[19:14] <lifeless> moin
[20:42] <ricotz> please, could someone restart https://launchpad.net/~xorg-edgers/+archive/ppa/+build/1945560 which is stuck
[20:46] <lifeless> ricotz: I don't think anyone with that access is around yet; you might try asking in #launchpad which is the support channel and has different people in it.
[21:00] <ricotz> lifeless, thanks
[21:19] <mwhudson> good morning
[21:47] <lifeless> whats that thing where you can get a librarian running before the test suite starts and use it ?
[21:48] <mwhudson> LP_PERSISTENT_TEST_SERVICES=1 ?
[21:48] <mwhudson> possibly spelt a bit differently
[21:52] <lifeless> hmm
[21:52] <lifeless> actually
[21:52] <lifeless> what I mean is
[21:53] <lifeless> HTF do I debug the librarian daemon
[21:53] <mwhudson> oh
[21:54] <mwhudson> lifeless: appears to be ./bin/start_librarian
[21:54] <lifeless> heres the scenario
[21:54] <lifeless> oh thanks
[21:55] <lifeless> I really need to track down why I get leaked processed 1/3 test runs
[21:56] <lifeless> mwhudson: so, I guess I need to set a config variable too ?
[21:56] <lifeless> what I want to do is: run some tests that make the librarian 500; with pdb on the librarian
[21:57] <mwhudson> lifeless: afaik know, LP_CONFIG defaults to development
[21:57] <mwhudson> aah
[21:57] <mwhudson> hm
[21:57] <mwhudson> s/know/no/ yay for homonym substitution
[21:57] <mwhudson> lifeless: maybe make start_librarian LP_CONFIG=testrunner
[21:58] <mwhudson> then run the tests you care about with LP_PERSISTENT_TEST_SERVICES=1 set  ?
[21:58] <mwhudson> would work mostly by chance i guess, but might work
[21:58] <lifeless> hah
[21:58] <lifeless> Daemons cannot log to stdout, exiting
[21:58] <mwhudson> :(
[21:58]  * lifeless files a bug
[21:59] <mwhudson> figure out what start_librarian does, do that but add -n to the twistd arguments
[22:00] <mwhudson> lifeless: it may be easier to do make run_all and recreate the tests by hand
[22:00] <mwhudson> (or not, depending on circs)
[22:01] <lifeless> it was entertaining finding this in the librarian
[22:01] <lifeless> raise LookupError
[22:23] <thumper> morning people
[22:50] <wgrant> mwhudson: Homophone! Not homonym!
[22:50] <wgrant> But anyway, morning.
[22:51] <thumper> morning wgrant
[22:51] <wgrant> How're things in NZ after the earthquake?
[22:51] <lifeless> wgrant: we're still here.
[22:51] <lifeless> chc is a bit fucked up
[22:51] <wgrant> Yeah, so it seems...
[22:52] <lifeless> we got out of there 2 weeks before the quake; good timing if I do say so myself
[22:52] <wgrant> Er, yes.
[22:53] <lifeless> wgrant: private librarian stuff is just about gtg
[22:53] <lifeless> https://code.edge.launchpad.net/++oops++/~lifeless/launchpad/private-librarian/+merge/31020 if you're interested in it
[22:54] <wgrant> I might remove the ++oops++ :)
[22:54]  * thumper goes to make a coffee
[22:54] <wgrant> lifeless: Oh, going straight to multiple domains?
[22:54] <wgrant> That's great.
[22:56] <wgrant> It would be nice if access with an invalid token would redirect to the webapp to get a new token.
[22:56] <wgrant> But LFAs don't have enough context :(
[22:56] <wgrant> Ah, I see you've already discussed that.
[23:01] <wgrant> lifeless:
[23:01] <wgrant> 480	+When the context file is a restricted `LibraryFileAlias`, traversal causes an
[23:01] <wgrant> 481	+access token to be allocated and a redirection to https on a unique domain to
[23:01] <wgrant> 482	+be issued.
[23:02] <wgrant> In that test, can you unelide the 'i....restricted'?
[23:02] <wgrant> Otherwise it's not obvious then that anything's different about the URL.
[23:02] <wgrant> And since it's meant to be documentation, that seems like a bad thing.
[23:04] <lifeless> that test file is unit tests masquerading as docs
[23:04] <wgrant> Ah.
[23:04] <thumper> lifeless: got time to chat?
[23:04] <lifeless> thats on line 465 for me
[23:04] <lifeless> thumper: sure
[23:04] <lifeless> skype?
[23:04] <thumper> lifeless: yep
[23:05] <wgrant> lifeless: Ah, I didn't have the latest rev.
[23:11] <wgrant> lifeless: So it doesn't actually check the domain?
[23:12] <lifeless> right
[23:12] <wgrant> Not a huge fan, but I guess it's OK.
[23:13] <lifeless> wgrant: if there is  ahole we can fix it, but I think its ok
[23:13] <wgrant> It just lets people make mistakes without noticing.
[23:14] <wgrant> Hm, actually, it might be dangerous.
[23:14] <wgrant> Yes, it is.
[23:15] <wgrant> I think.
[23:15] <wgrant> I forget the exact cross-window security restrictions....
[23:15] <wgrant> Why can't this be simple? :(
[23:16] <mwhudson> wgrant: ah right
[23:18] <wgrant> I'm also not sure how browsers treat Referer when leaving an HTTPS URL for another HTTPS domain.
[23:18] <wgrant> The RFC only says that they shouldn't send it when going non-secure.
[23:18] <wgrant> Not cross-domain.
[23:21] <lifeless> wgrant: what would the attack be
[23:22] <lifeless> I've asked kees to review as well
[23:23] <wgrant> lifeless: Somebody visits a private file. I can come along and send them to a page which lives on that same domain. Now, I'm not entirely sure how to get the other URL, but we've now bypassed cross-domain restrictions, so it needs thought.
[23:23] <wgrant> Complicated :(
[23:24] <wgrant> Ah, I know.
[23:25] <wgrant> I know that somebody has access to a private file. I know its webapp URL, LFA ID and filename.
[23:25] <wgrant> I get an HTML file into a library file, and send them a URL to it on the target LFA's domain.
[23:25] <wgrant> That page uses an iframe to go to the webapp URL (thus holding a reference to the window).
[23:26] <wgrant> Once it gets to the webapp, my nasty page can't access the iframe (because it's on a different domain).
[23:26] <wgrant> But the webapp will then redirect back to the file, on my domain.
[23:26] <wgrant> Once it's back on my domain, I can access properties of the window (including its URL).
[23:26] <wgrant> I think that should work.
[23:28] <lifeless> so, concretely
[23:29] <lifeless> you file a private bug that they will look at
[23:29] <lifeless> it has an attachment which will look up some other thing via the attach you describe above
[23:29] <lifeless> and to block it we need to make the domains line up
[23:29] <wgrant> It doesn't need to be a private bug.
[23:29] <lifeless> s/attach/attack/
[23:30] <wgrant> I just need to get them to a librarian URL somehow.
[23:30] <wgrant> But yes, you need to ensure that the domains match.
[23:30] <lifeless> which means knowing the LFA, which is only shown if you have access
[23:30] <wgrant> lifeless: Oh really?
[23:30] <wgrant> It can be guessed.
[23:30] <wgrant> I've done so on a number of occasions :)
[23:30] <lifeless> ok
[23:31] <lifeless> I'm trying to estimate the risk if we:
[23:31] <lifeless>  - deploy roughly what we have today
[23:31] <lifeless>  - check the request path to be sure Host is preserved
[23:31] <lifeless>  - enhance it to enforce domain matching in a future revision
[23:31] <wgrant> "to be sure Host is preserved"?
[23:32] <lifeless> the request path for the private librarian is: client -> apache -> squid -> librarianN
[23:32] <lifeless> I'm not entirely sure the host header will be getting through untouched *right now* because we've never depended on it.
[23:33] <wgrant> Oh, request path as in path of the request.
[23:33] <wgrant> Not the path attribute of the request.
[23:33] <wgrant> Right.
[23:33] <lifeless> to enforce the domain matches the LFA id on all requests, we need to make sure its preserved
[23:33] <wgrant> Yep.
[23:33] <lifeless> wgrant: I'd like to get an iteration of this live on thursday
[23:34] <lifeless> wgrant: we have essentially 2 days to get all the kinks out, or to defer some stuff.
[23:34] <wgrant> We should also check out how browsers handle Referer.
[23:34] <wgrant> I can't find any explicit mention of them behaving sanely :(
[23:34] <lifeless> I don't want to deploy a badly broken system, but if it is better than what we had, and relatively low risk, it might be ok for a week or two
[23:34] <lifeless> wgrant: assume insanity
[23:34] <wgrant> lifeless: I am.
[23:36] <wgrant> (the issue here is that links from private files to external HTTPS sites may reveal the file to the target site)
[23:36] <lifeless> wgrant: yes. I don't have any ideas how to do that other than having a cookie setting service.
[23:37] <lifeless> wgrant: and it would still reveal the existence of the files
[23:37] <lifeless> wgrant: OTOH a file can only shoot itself in the foot
[23:37] <wgrant> Yeah, the solution I thought of was to have the tokenised URL set a cookie then redirect to something without the token.
[23:38] <wgrant> But grrrr.
[23:38] <wgrant> Stupid web.
[23:38] <lifeless> wgrant: yeah, 'cookie setting service'. meep sucky.
[23:39] <lifeless> wgrant: but back to /now/ : is the current thing fatally flawed, or something we could iterate on over a couple weeks.
[23:39] <lifeless> deploy + iterate, that is.
[23:39] <lifeless> I'm personally fine with attachments that shoot themselves in the foot.
[23:39] <wgrant> lifeless: I'm not comfortable making a statement either way.
[23:39] <wgrant> Right, that's probably OK.
[23:39] <wallyworld_> morning
[23:40] <wgrant> The attack I outlined earlier is my concern.
[23:40] <wgrant> But that *in combination* with the shooting-themselves-in-the-foot is pretty bad.
[23:40] <wgrant> If Referer is indeed sent.
[23:40] <thumper> wallyworld_: morning
[23:41] <wgrant> Difficulties I see with implementing the domain restriction:
[23:41] <wgrant>  - As you say, the request path may not preserve Host.
[23:41] <wgrant>  - We might have to make launchpad.dev exempt, since otherwise we need wildcard /etc/hosts....
[23:43] <lifeless> right, there is a comment in the tests about that ;)
[23:44] <lifeless> we can actually test the librarian without it
[23:44] <lifeless> (connect on ip, host header passed to librarian)
[23:44] <wgrant> Yep.
[23:44] <lifeless> and we could check that private bug /urls/ are of the right shape, not actually connect.
[23:45] <lifeless> attachment urls, I mean.
[23:45] <lifeless> do a non-follow-redirect request
[23:45] <wgrant> (also, are you going to be able to get a cert in time?)
[23:46] <lifeless> wgrant: its the top ticket in the LP queue.
[23:46] <lifeless> wgrant: also on the MP it says 'have a thing to allow it to be enabled post rollout'
[23:46] <wgrant> Ah, good.
[23:46] <wgrant> I didn't actually read the MP description.
[23:46] <wgrant> Just the diff.
[23:46] <lifeless> specifically I'd like to get the code out there
[23:46] <lifeless> post rollout hernias addressed.
[23:48] <lifeless> then get a manually inserted TLT, check with wget it works, then enable the appserver code.
[23:48] <wgrant> I prefer to read the code first, so my interpretation isn't incorrectly influenced by the description.
[23:48] <wgrant> Right.
[23:48] <wgrant> Wait, TLT?
[23:48] <lifeless> time limited token
[23:48] <wgrant> Oh, right.
[23:48] <wgrant> I guess this is what feature flags are for.
[23:49] <lifeless> yes
[23:49] <wgrant> Handy.
[23:49] <lifeless> they're a bit rought still
[23:49] <lifeless> but will do the job
[23:56] <lifeless> wgrant: http://www.geonet.org.nz/images/news/2010/Fault_0564.jpg
[23:56] <lifeless> http://www.geonet.org.nz/news/article-sep-4-2010-christchurch-earthquake.html
[23:58] <wgrant> Ow.