[00:49] lifeless, do you know of anyone who has a .testr.conf for nose? [00:49] lifeless, I know about the TestID plugin for nose, but using testr itself would be nice [01:13] lifeless, turns out to be pretty simple: http://pastebin.ubuntu.com/488515/. But for some reason it will not pick up the list of failing tests. [01:14] ah ha! [01:18] lifeless, you need to redirect STDERR for it to work with nose. This /almost/ works as expected: http://pastebin.ubuntu.com/488516/ [01:57] mars: sweet [01:58] no idea why it would be writing subunit to stderr. [01:58] still, mine not to wonder why [05:55] I've no idea how its getting negatives [05:57] finish sets duration to a timedelta of 'NOW - start' [05:57] start is set to NOW [05:57] NOW - NOW = 0, so the duration should be the change in NOW between the two calls being made. [05:57] jtv: & [05:57] jtv: ^ [05:58] I wonder if its an errorlog bad-responsibilities thing [05:59] lifeless: you're sure only one clock is involved? [05:59] yes [05:59] the durations go down almost linearly as the offset increases [05:59] context? [05:59] so something is substracting duration again [06:00] bug 630612 [06:00] <_mup_> Bug #630612: Complete b0rkage of oops timing info [06:00] e.g. https://lp-oops.canonical.com/oops.py/?oopsid=1708EB636 [06:00] Hmm it's definitely not clock skew. [06:00] add offset to the times and its right [06:00] I think [06:01] Are you accidentally throwing the baseline time into the subtraction? [06:01] no [06:01] definitional issue [06:01] putting up a patch [06:01] As in, finish - start - baseline? [06:03] no, its that it doesn't want a duration [06:03] the old code wasn't tested [06:03] and I misunderstood what the disk format was meant to represent [06:07] https://code.edge.launchpad.net/~lifeless/launchpad/oops/+merge/34635 [06:08] thumper: are you around, perchance ? [06:08] not really [06:08] whats up? [06:09] I broke oopses on edge; fix is trivial, PQM is closed, needs a release-critical stamp [06:09] which is you, or gmb in about 3-4 hours [06:09] lifeless: that mp up? [06:09] https://code.edge.launchpad.net/~lifeless/launchpad/oops/+merge/34635 [06:09] by broke, they work, but you need to think like a pretzel to analyse them [06:09] diff not there yet [06:10] * lifeless bets its hung [06:10] I've pasted a diff in [06:11] lifeless: the diff and comment don't really help me understand it [06:11] but if you are sure it is right [06:11] I'll rc it [06:12] thumper: I can explain it pretty quickly [06:12] go then [06:12] look at this oops [06:12] https://lp-oops.canonical.com/oops.py/?oopsid=1708EB636 [06:12] * thumper is supposed to be making dinner [06:12] specifically the sql log [06:12] notice how the durations go negative [06:12] the actual duration is the reported duration + the start offset. [06:12] aye [06:13] my patch adds the start offset in the generation code [06:13] ok [06:13] I misunderstood what the disk format was meant to have in it. [06:13] done [06:13] thanks [06:13] jtv: care to provide the code review stamp that will be wanted by ec2land ? [06:14] lifeless: since you're phrasing it like I'll have a meaningful role in the process, sure [06:15] This won't hit sundays rollout, but I can nurse it into devel tonight and ask spm to trigger a reroll of edge monday am [06:15] sorry about breaking it [06:21] jtv: I think you do have a meaningful role - there are three people that have read the entire change here: you, me, mwhudson [06:21] jtv: I can't think of anyone better suited to review this (trivial) patch [06:23] Well, it's reviewed. [06:24] thank you [06:24] The only way I could find to do that was to request a review from myself. [06:24] oh [06:24] all you have to do is type in the comment box [06:24] I could've added an Approve vote, but not with a specific review type. [06:24] and change the drop-down to approve/needs info etc [06:24] And you want the "code" review type for "ec2 land." [06:24] ah [06:24] I think a default type == code, doesn't it ? [06:27] jtv: thanks for catching this [06:27] I would have when I got a chance to look, but I'm glad to catch it as early as possible [06:44] I so want a button I can push. [06:44] which will deploy. [06:47] * mwhudson is tempted to say that lifeless almost certainly owns a device which can cause a deployment by pressing lots of buttons in the right order [06:49] lifeless: i've just read set_request_timeline :) [06:49] s/:)/:(/ [06:49] mwhudson: I would deeply deeply love to have that better.. [06:50] mwhudson: I'm going to be putting it fairly high up in my hygiene requests for foundations I think; can't build on a shaky base. [06:52] DistributionSourcePackage:+addquestion is looking pretty unhealthy [06:54] lifeless: did you do any coding towards bug 623199 ? [06:54] <_mup_> Bug #623199: scripts do not establish valid zope partiticipations [06:55] mwhudson: nothing reusable [06:55] mwhudson: I scrapped it as a learning experience [06:55] i could have a hack now, but i guess an hour is probably extremely optimistic to get something useful done [06:55] lifeless: what did you learn? [06:56] I learnt that our scripts code is horribly confused about what they do [06:56] :( [06:56] I am more and more of the opinion we want impersonation [06:56] we want something to be able to: [06:56] - run async [06:57] - use the API to do shit [06:57] - do it on behalf of a user that originated the work [06:57] I think the second line, when written 'use SQL to do shit' [06:57] should mean nearly-no-code-changes, ideally. [06:58] e.g. scripts should setup a participation of the user who the work is on behalf of [06:58] well i can see that would be good [06:58] this is obviously only applicable to deferred-work-scripts [06:58] but can't we make canonical.launchpad.webapp.adapter less terrible without that? [06:58] others like the PPA access token updater are conceptually different [06:58] mwhudson: sure [06:59] lifeless: and by run async, you mean in the context of appserver requests? [06:59] no [06:59] not twisted scripts that somehow process more than one job at once? [06:59] badly phrased [06:59] 'run out of step with appserver stuff' [06:59] ok, that's what i thought you meant [06:59] maybe for a single request, maybe much later, and all things inbetween [06:59] i phrased it badly too [07:00] :) [07:00] anyhow [07:00] yes, we can make adapter better [07:00] I think your plan is a good one: [07:00] - a specific interface which needs the following characteristics: [07:01] - adapter and other code like it (timelines, featureflags, permission checking) can rely on [if missing the error/fail appropriate;y] [07:01] - HttpRequest implements it [07:02] - something for scripts implements it [07:02] we may want the ability to push-and-pop the contextual-lookup for these objects [07:02] example: [07:03] checkwatches starts up, it needs to (picking one such thing) get the timeline for it as a whole and use that while it queries the DB for watches to update [07:03] it may then want to, per watch, push a new context, which will get the db queries, errors, http client times, mail sending times, for a single watch. [07:04] well [07:04] there is newInteraction and restoreInteraction [07:04] sure [07:04] so zope already supports this to some extent [07:04] I'm trying not to talk impl [07:04] ok [07:04] you know whats available much better than I [07:04] i only learnt about restoreInteraction a couple of weeks ago :-) [07:04] see [07:05] you're a couple of weeks ahead of me :P [07:05] thats a big fraction of my time in this job :> [07:05] well, if it took me three+ years and you a few months, that's a good sign all round i think [07:05] ha! [07:06] i think launchpad suffered for a time for not having any one who really got zope [07:06] I've been listening in the corners all this time [07:06] so we should make sure we hang on to gary and benji :-) [07:06] mmm, we *started* with serious zopers. [07:07] anyhow, we're going well now [07:07] and I can see a path to having headroom to really tackle things. [07:08] yeah [07:08] that's good [07:08] so webapp adapter [07:08] I can - I have - described what I think we need in broad terms. [07:09] building on your description on the list [07:09] what it needs now is someone to implement it and migrate a couple of scripts over, such that we can see it has legs. [07:10] you could probably do the start in an hour [07:11] I dunno about getting into the meat [07:11] time for me to put on the war of the worlds and ratchet up the private librarian refactoring [07:13] fair enough [07:21] i think i'm going to think about linaro stuff instead :) [07:29] \o/ next rollout will be tracking email/librarian/memcache times [07:29] badly [07:29] but tracking them [09:44] jtv: bug 629921 might entertain you [09:44] <_mup_> Bug #629921: Archive:+packages with empty name search does like '%%' search. [09:59] What's the likelyhood of finding someone capable of bouncing codebrowse on a Sunday? :-/ [10:31] maxb: low to middling [10:32] hmm, whats the recommended url parsing lib for lp code [11:10] lifeless: lazr.uri, probably. [11:11] wgrant: do you know of any use for https urls on librarian files? [11:11] (the current system, I mean) [11:18] lifeless: They're used in the webapp. [11:18] To avoid insecure content warnings. [11:18] eg. project icons. [11:18] blah [11:18] * lifeless rethinks part of this [11:19] the docs could at least be a little less daft about how they explain it [11:27] man the layers are messy [11:27] I think its been refactored and docstrings not changed. [11:32] lifeless, So, the current build failure is because the checked-in wadl is out of sync with what's generated by LP. I'm regenerating the on-disk wadl and checking it in; that should fix the breakage. [11:32] heh [11:33] so are we meant to do that always? How do we tell if a change is incompatible? [11:33] could you perhaps file a bug asking these things of foundations :) [11:33] actually [11:34] gmb: can you please disable the test. [11:35] lifeless, You have a good point. I think I can shed some light on the reason for the test anyway: [11:36] 1. Before we had checked-in WADL, there were always bzr conflicts because people would accidentally check in the apidoc directory (which would be created if it didn't exist already) [11:36] 2. Therefore, it was decided to check in the WADL to prevent those conflicts. [11:36] 3. Trouble was that WADL generation took a long time and unless it's --forced it won't overwrite the extant files. [11:37] So the test is there to prevent us from having something broken rolled out. [11:37] lifeless, I don't think we should disable the test unless we're going to stop having the WADL checked-in. [11:37] gmb: I've replied in the thread [11:38] gmb: but lets check my logic. [11:38] we have two branches. [11:38] stable, db-devel. [11:38] both receive API changes. [11:38] whats going to happen in a cycle after both have had -any- API change. [11:39] lifeless, Your reasoning is sound. [11:39] lifeless, Okay, I agree; I'll disable the test. [11:39] Perhaps revert the merge that added it to restore the old logic, whatever that was. [11:39] Checking in WADL seems somewhat... odd. [11:39] I don't know enough of the guts to suggest the right thing to do. [11:39] I believe there was an additional desire to prevent API regressions by making people thing. [11:39] s/thing/think/ [11:40] On reflection, I don't think the WADL is human readable enough for developers to do that routinely. [11:40] lifeless, I think that test has been around for a while, actually. [11:40] gmb: really ? [11:41] lifeless, Yes. THough I'm not certain. I'll check now. [11:41] I though benji landed it late last weke [11:41] \o/ we should be getting librarian stuff in oops now. /me goes to try [11:42] lifeless, Oh, right. For some reason I thought it was something that had been kicking around for a while. In my head, I was blaming mars ;) [11:42] the motivations are good [11:42] needs some more glue to work well [11:44] lifeless, You're right; it landed on devel the week before last. [11:45] I'll revert the merge(s). [11:46] thanks [11:46] my sunday is fading fast [11:47] lifeless, Okay, no worries. I'll take care of this. [11:54] \o/ [11:55] something slightly screwy [11:56] 15ms to connect to librarian in the dc [11:56] and 0ms to get the diff down [11:57] 15ms is a bit slow [11:57] I swear PQM keeps adding things to the regex so that my submissions fail. [11:57] https://lp-oops.canonical.com/oops.py/?oopsid=1709EB904 for folk that can see [11:57] gmb: what did you try [11:57] and to what branch; they are funkily different [11:58] lifeless, Oh, I'm being facetious. For some reason it's asking for [ui=] as well as [testfix][r-c][rs]. [11:58] \o/ [11:58] what branch [11:58] devel [11:58] freaky [11:58] Indeed. [11:59] so this cycle [11:59] when questions goes beserk, I'll be able to point and laugh at email very very easily :P [11:59] * lifeless bets sending email is not cheap [12:02] lifeless: hi, your last comment on bug 620458 is a big surprise too me ;) the code from my last comment did not work a week ago, [12:02] <_mup_> Bug #620458: cannot access attachments of private bugs any more [12:03] thekorn: are you using production or edge? [12:03] and no, I'm not running the code in your datacenter [12:03] lifeless: works on both, maybe I was facing a different issue [12:03] perhaps [12:03] perhaps deryck and abel rolled back the privacy change [12:03] I'm working on the long term fix atm [12:03] hopefully we'll get it out in this rollout, and it will be faster after that. [12:04] lifeless: my way to reproduce this bug was always: "attachments are not accessible for private bugreports not reported by myself" [12:04] thekorn: well, I'll talk to deryck tomorrow night [12:04] gmb may know stuff now. [12:04] gnight y'all [12:04] great [12:04] good noight [12:05] thekorn, I'm afraid I don't know much about the private attachment stuff besides that there's still work ongoing. I'll speak to adeuring in the morning. [12:08] gmb: no problem, just wanted to give lifeless a quick answer to the question he had in his last comment [12:09] thekorn, Ah, okay, cool. [19:14] moin [20:42] please, could someone restart https://launchpad.net/~xorg-edgers/+archive/ppa/+build/1945560 which is stuck [20:46] ricotz: I don't think anyone with that access is around yet; you might try asking in #launchpad which is the support channel and has different people in it. [21:00] lifeless, thanks === Pilky_ is now known as Pilky [21:19] good morning === gmb` is now known as gmb [21:47] whats that thing where you can get a librarian running before the test suite starts and use it ? [21:48] LP_PERSISTENT_TEST_SERVICES=1 ? [21:48] possibly spelt a bit differently [21:52] hmm [21:52] actually [21:52] what I mean is [21:53] HTF do I debug the librarian daemon [21:53] oh [21:54] lifeless: appears to be ./bin/start_librarian [21:54] heres the scenario [21:54] oh thanks [21:55] I really need to track down why I get leaked processed 1/3 test runs [21:56] mwhudson: so, I guess I need to set a config variable too ? [21:56] what I want to do is: run some tests that make the librarian 500; with pdb on the librarian [21:57] lifeless: afaik know, LP_CONFIG defaults to development [21:57] aah [21:57] hm [21:57] s/know/no/ yay for homonym substitution [21:57] lifeless: maybe make start_librarian LP_CONFIG=testrunner [21:58] then run the tests you care about with LP_PERSISTENT_TEST_SERVICES=1 set ? [21:58] would work mostly by chance i guess, but might work [21:58] hah [21:58] Daemons cannot log to stdout, exiting [21:58] :( [21:58] * lifeless files a bug [21:59] figure out what start_librarian does, do that but add -n to the twistd arguments [22:00] lifeless: it may be easier to do make run_all and recreate the tests by hand [22:00] (or not, depending on circs) [22:01] it was entertaining finding this in the librarian [22:01] raise LookupError [22:23] morning people [22:50] mwhudson: Homophone! Not homonym! [22:50] But anyway, morning. [22:51] morning wgrant [22:51] How're things in NZ after the earthquake? [22:51] wgrant: we're still here. [22:51] chc is a bit fucked up [22:51] Yeah, so it seems... [22:52] we got out of there 2 weeks before the quake; good timing if I do say so myself [22:52] Er, yes. [22:53] wgrant: private librarian stuff is just about gtg [22:53] https://code.edge.launchpad.net/++oops++/~lifeless/launchpad/private-librarian/+merge/31020 if you're interested in it [22:54] I might remove the ++oops++ :) [22:54] * thumper goes to make a coffee [22:54] lifeless: Oh, going straight to multiple domains? [22:54] That's great. [22:56] It would be nice if access with an invalid token would redirect to the webapp to get a new token. [22:56] But LFAs don't have enough context :( [22:56] Ah, I see you've already discussed that. [23:01] lifeless: [23:01] 480 +When the context file is a restricted `LibraryFileAlias`, traversal causes an [23:01] 481 +access token to be allocated and a redirection to https on a unique domain to [23:01] 482 +be issued. [23:02] In that test, can you unelide the 'i....restricted'? [23:02] Otherwise it's not obvious then that anything's different about the URL. [23:02] And since it's meant to be documentation, that seems like a bad thing. [23:04] that test file is unit tests masquerading as docs [23:04] Ah. [23:04] lifeless: got time to chat? [23:04] thats on line 465 for me [23:04] thumper: sure [23:04] skype? [23:04] lifeless: yep [23:05] lifeless: Ah, I didn't have the latest rev. [23:11] lifeless: So it doesn't actually check the domain? [23:12] right [23:12] Not a huge fan, but I guess it's OK. [23:13] wgrant: if there is ahole we can fix it, but I think its ok [23:13] It just lets people make mistakes without noticing. [23:14] Hm, actually, it might be dangerous. [23:14] Yes, it is. [23:15] I think. [23:15] I forget the exact cross-window security restrictions.... [23:15] Why can't this be simple? :( [23:16] wgrant: ah right [23:18] I'm also not sure how browsers treat Referer when leaving an HTTPS URL for another HTTPS domain. [23:18] The RFC only says that they shouldn't send it when going non-secure. [23:18] Not cross-domain. [23:21] wgrant: what would the attack be [23:22] I've asked kees to review as well [23:23] lifeless: Somebody visits a private file. I can come along and send them to a page which lives on that same domain. Now, I'm not entirely sure how to get the other URL, but we've now bypassed cross-domain restrictions, so it needs thought. [23:23] Complicated :( [23:24] Ah, I know. [23:25] I know that somebody has access to a private file. I know its webapp URL, LFA ID and filename. [23:25] I get an HTML file into a library file, and send them a URL to it on the target LFA's domain. [23:25] That page uses an iframe to go to the webapp URL (thus holding a reference to the window). [23:26] Once it gets to the webapp, my nasty page can't access the iframe (because it's on a different domain). [23:26] But the webapp will then redirect back to the file, on my domain. [23:26] Once it's back on my domain, I can access properties of the window (including its URL). [23:26] I think that should work. [23:28] so, concretely [23:29] you file a private bug that they will look at [23:29] it has an attachment which will look up some other thing via the attach you describe above [23:29] and to block it we need to make the domains line up [23:29] It doesn't need to be a private bug. [23:29] s/attach/attack/ [23:30] I just need to get them to a librarian URL somehow. [23:30] But yes, you need to ensure that the domains match. [23:30] which means knowing the LFA, which is only shown if you have access [23:30] lifeless: Oh really? [23:30] It can be guessed. [23:30] I've done so on a number of occasions :) [23:30] ok [23:31] I'm trying to estimate the risk if we: [23:31] - deploy roughly what we have today [23:31] - check the request path to be sure Host is preserved [23:31] - enhance it to enforce domain matching in a future revision [23:31] "to be sure Host is preserved"? [23:32] the request path for the private librarian is: client -> apache -> squid -> librarianN [23:32] I'm not entirely sure the host header will be getting through untouched *right now* because we've never depended on it. [23:33] Oh, request path as in path of the request. [23:33] Not the path attribute of the request. [23:33] Right. [23:33] to enforce the domain matches the LFA id on all requests, we need to make sure its preserved [23:33] Yep. [23:33] wgrant: I'd like to get an iteration of this live on thursday [23:34] wgrant: we have essentially 2 days to get all the kinks out, or to defer some stuff. [23:34] We should also check out how browsers handle Referer. [23:34] I can't find any explicit mention of them behaving sanely :( [23:34] I don't want to deploy a badly broken system, but if it is better than what we had, and relatively low risk, it might be ok for a week or two [23:34] wgrant: assume insanity [23:34] lifeless: I am. [23:36] (the issue here is that links from private files to external HTTPS sites may reveal the file to the target site) [23:36] wgrant: yes. I don't have any ideas how to do that other than having a cookie setting service. [23:37] wgrant: and it would still reveal the existence of the files [23:37] wgrant: OTOH a file can only shoot itself in the foot [23:37] Yeah, the solution I thought of was to have the tokenised URL set a cookie then redirect to something without the token. [23:38] But grrrr. [23:38] Stupid web. [23:38] wgrant: yeah, 'cookie setting service'. meep sucky. [23:39] wgrant: but back to /now/ : is the current thing fatally flawed, or something we could iterate on over a couple weeks. [23:39] deploy + iterate, that is. [23:39] I'm personally fine with attachments that shoot themselves in the foot. [23:39] lifeless: I'm not comfortable making a statement either way. [23:39] Right, that's probably OK. [23:39] morning [23:40] The attack I outlined earlier is my concern. [23:40] But that *in combination* with the shooting-themselves-in-the-foot is pretty bad. [23:40] If Referer is indeed sent. [23:40] wallyworld_: morning [23:41] Difficulties I see with implementing the domain restriction: [23:41]