[00:19] do we still use the keyring trust analyzer ? [00:28] lifeless: I doubt it and I hope not. [00:28] But it doesn't seem to log to scriptactivity, so we'll never know. [00:28] hmm [00:28] where is that cronscripts branch [00:29] It's not in there. [00:29] ah, you checked ? [00:29] wgrant@lamuella:~/launchpad/lp-production-crontabs$ bzr grep email-clus [00:29] wgrant@lamuella:~/launchpad/lp-production-crontabs$ [00:31] It was added in 2005 and hasn't really been touched since. [00:31] I blame jamesh. [00:31] Maybe he knows something. [00:31] I suspect it's been obsolete for 5 years, but we'd best check. [00:31] I think I had something to do with it too :P [00:31] I'll just delete it. [00:31] You appear too, yes. [00:31] * robert.collins@canonical.com/launchpad--keyring-trust-analyzer--0--patch-1 [00:31] snapshotting current pgp work [00:31] But jamesh is all the later stuff. [00:31] IIRC I did the plumbing [00:32] Gets rid of stupid tests from canonical.launchpad, too. [00:32] Kill it! [00:36] Ah, I didn't do too badly last week. [00:36] ? [00:36] My stale appserver pid file is from before EOD on the Friday before! [00:37] class LogCollector(logging.Handler): [00:37] -yet- -another- -yes- [00:37] Heh. [00:37] They're mostly cleaned up! [00:44] lifeless: I guess there wasn't much fdt progress last week? [00:47] you guess well [00:52] not as much code as I hoped [00:52] 7 files changed, 750 deletions(-) [00:52] :( [00:59] can has revooo https://code.launchpad.net/~lifeless/launchpad/bug-830789/+merge/72371 [01:02] Huh, only three gpghandler methods? [01:02] indeed [01:03] I'm not sure what to do with zeca [01:03] it seems massively overkill for it to be a twisted daemon [01:04] but I'd hate to have a transitive dependency on twisted for a web.py microservice [01:04] Probably. [01:04] so I need to either rewrite it, or have two entirely separate packages with gpg support stuff [01:04] I'm thinking rewrite [01:08] lifeless: Approved, but with lint. [01:09] hah [01:09] at least its valid [01:09] Yes. [01:10] wgrant: did you see privmsg ? [01:12] Missed that, sorry. [01:23] wgrant: I don't think we ever put the keyring trust analyzer code into production (at least, we didn't while I was working on LP) [02:28] lifeless! [02:28] You deleted the pygpgme sourcecode! [02:28] Yay. [02:28] Burn! [02:30] Re-add it so we can delete it again! [02:36] wgrant: yes, I did. [02:38] wheee 1800 OOPS-2059AY24 POFile:+translate [02:43] wgrant: It moves to an egg [02:43] StevenK: Yes. [02:43] But at least it's not a patched branch in sourcecode. [02:44] Bah, why do I still have a sourcecode/shipit directory [03:58] I wish the test suite wouldn't leave my /tmp full of launchpadlib caches and my swap full of librarians. [04:00] jtv: you're experiencing librarian leakage? [04:00] Yes. [04:00] * jtv had no idea such a word existed [04:00] grah. I thought we'd squashed that [04:00] jtv: does that happen when no failures happen? [04:01] or only when you have failures / errors? [04:01] No idea. Don't know what the lifetime of a librarian is currently supposed to be, so I have no basis for comparison. [04:02] I can tell you that the launchpadlib caches and the createdb templates (whatever those may be) are leaking with successful test runs. [04:02] its meant to be the lifetime of the layer [04:02] the createdb templaes - launchpad_ftest_template_1234 ? [04:03] or launchpad_ftest_1234? [04:07] Trying it again now. [04:08] the former has a lifetime of test-runner-process [04:08] the latter is a lifetime of one-test [04:08] A limited test run just leaked lp.createdb.launchpad_ftest_template_13091 and lp.createdb.launchpad_ftest_template [04:08] so the leak points for them are different [04:08] jtv: oh, these are files in /tmp ? [04:08] Yes [04:08] I would really like to squash the test suite creating odd files [04:08] ok, please file a bug on them leaking; [04:09] they exist to workaround postgresql behaviour on concurrent CREATE DATABASE calls [04:09] StevenK: so would I. Passionately. [04:09] the former we can fix leaks of easily - when the test runner cleansup the temp template db it can remove the lock [04:09] One of the archiveuploader tests creates a strange file [04:09] the latter file is a bit harder to nuke safely [04:10] lifeless: if the leak points are different, should they be the same bug? [04:10] jtv: could go either way [04:10] StevenK: a big source of leaks on publisher tests for me has been the lack of publisher config pointing to temp directories. [04:11] I'm trying a passing test that uses the librarian now, to see if that leaks. [04:11] jtv: I'm not really concerned about files in temporary directories -- I'm concerned about tests that dump random files into the source tree [04:12] StevenK: yes, that's what I'm talking about. [04:12] steven@liquified:~% ls -1 /tmp/lp.createdb.* | wc -l [04:12] 105 [04:12] RARGH [04:12] It's a good thing suspend and hibernate don't work, so I reboot regularly anyway. [04:13] lifeless: a successful test with the librarian just now did not leak a librarian. [04:13] thanks [04:13] I wish I had time deal with that storm problem that broke the fake librarian. :/ [04:16] lifeless: bug 830845 [04:16] <_mup_> Bug #830845: Tests leak "lp.createdb" files in /tmp < https://launchpad.net/bugs/830845 > [04:17] I bet it's nascentupload [04:18] I think I just got it from running registry browser tests. [04:18] I can recall one from archiveuploader and one from code tests [04:19] lifeless: these look like empty marker files. Would it help debugging to dump a stack trace into them? [04:19] Hmm, the unlock is in a finally: :/ [04:21] lifeless: Do you understand sinzui's comment in bug #829105? [04:21] <_mup_> Bug #829105: Able to target to release that I cannot upload to

< https://launchpad.net/bugs/829105 > [04:22] jtv: no, they are just lock files [04:22] jtv: we use a lock to prevent calling CREATE DATABASE with the same template concurrently, because doing that breaks postgresql [04:22] Exactly. If you need to know where they are leaked from, it might help to know more about where they are created. [04:23] jtv: there is only one place that makes them [04:23] jtv: the stacktrace will be identical every time [04:23] OK [04:24] the one without a number in it is made when we clone launchpad_ftest_template to a per-runner template [04:24] that reduces contention on db cloning in parallel test environments [04:25] the one with a number in it is made when we clone each concrete test db (because the same code runs each time) [04:25] wgrant: which unlock ? [04:26] wgrant: I think sinzui means 'before the recent retargeting work you could not do this' + 'if someone does it the impact is severe because of the inability to delete the task' [04:27] lifeless: The recent retargeting work is unrelated. [04:27] lifeless: The lp.createdb unlock. [04:27] wgrant: I can't remember which lock system I used [04:27] wgrant: but most won't delete the lock because doing so can be racy [04:27] Ah. [04:28] e.g. with a lockdir, the lockdir itself is racy-as, but the object so locked can be locked reliably [04:29] Morning! [04:30] Hi nigelb. [04:30] Can haz review? https://code.launchpad.net/~nigelbabu/launchpad/4595-upgrade-bug-linking/+merge/71575 [04:30] Morning wgrant [04:30] (actually, is it afternoon for you?) [04:30] It is 14:30 [04:31] Ah [04:31] not its not, its 16:30 [04:32] woah, didn't know you were living that far ahead in the future. [04:32] New Zealand doesn't exist. [04:32] nigelb: I think 'Checks if links of the form /bugs/100' should be re-worded [04:32] <_mup_> Bug #100: uploading po file overwrites authors list

< https://launchpad.net/bugs/100 > [04:33] ah, yes. [04:33] I had a branch that set $TEMPDIR etc. per test, but then made the mistake of trying to enforce stuff to clean up temp files and it never landed :-( [04:34] StevenK: At, remove the first 'if'? :) [04:34] *Aah [04:34] That sounds better [04:35] jtv: Your em dashes offend me. [04:35] Good. [04:39] Dear bzr push, GO FASTER [04:45] No problem here with the UK, but the US? [04:47] StevenK: well I found out why my dputs to dogfood didn't work. Next questions are why, and what to do about it. In dput config section [dogfood], "incoming = %(dogfood)s/ubuntu" translates to "ubuntu". [04:47] StevenK: updated. [04:48] jtv: If you just ran dput dogfood , then that would explain it [04:49] That's what I've always been told to run. Is it wrong? [04:51] It can be. Where do you want your uploads to go? [04:52] If you say "Onto dogfood", I'll slap you [04:54] “Wherever on dogfood a test upload for an Ubuntu series is supposed to go.“ [04:55] * jtv thinks he dodged that slap rather nicely [04:55] heh [04:56] jtv: It depends if you are uploading to a ppa as well, you see [04:56] I'm not [04:56] As far as I know. [04:56] I'm trying to upload to Ubuntu Ridiculous Rat [04:58] I think /ubuntu should be okay then [04:59] Not /srv/launchpad.net/ubuntu-queue/ubuntu or somesuch? [04:59] I mean, the poppy log suggests it's trying to write to /ubuntu [05:00] (Unless it's not being very clear, which is also possible) [05:00] exceptions.OSError: [Errno Path not allowed:] /ubuntu/farewell_1.0-1.dsc.tmp.1313988369.200371027.15260.1143614958 [05:01] Hm [05:01] Something called rootpath seems to be empty for some reason. [05:01] But it's got to be something on my end, because it was working for Raphaël. [05:01] It's been too long since I dealt with poppy [05:02] Oh, no, rootpath is okay; it's just that the path does not begin with it. [05:03] Maybe it needs a restart. One of the line numbers in the traceback was off by one. [05:03] jtv: My incoming = %(dogfood)s, and I think for distro uploads I use dogfood:ubuntu [05:04] I was told to set incoming = %(dogfood)s/ubuntu (and then I use just dogfood, not dogfood:ubuntu) [05:04] I see we have a script for starting poppy on dogfood. How do we stop poppy though? [05:05] kill; [05:06] There's no pidfile? [05:07] ps aux | grep poppy [05:07] Grr [05:11] there should be a pid file [05:12] It could be created on prod by start-stop-daemon [05:12] But poppy is a twistd. [05:12] There should be one. [05:12] But ps is easier than tracking it down. [05:13] It's been running for 5 minutes, so I guess jtv has restarted it [05:13] Yup [05:13] But I'd much prefer to automate it. [05:13] StevenK: One moreround of reviews? :) [05:14] wgrant: twistd generates pidfiles by default? I didn't see anything in /var/run [05:14] nigelb: I didn't review the last one? [05:14] It does, yes. The default is ./twistd.pid [05:14] Which has to be the most useless thing EVER [05:14] /var/tmp/poppy/poppy.pid! [05:15] StevenK: Oh. If you're busy, should I wait for OCR? [05:15] jtv: Which is wrong [05:15] Oh [05:15] It's definitely got the wrong pid in it, yes [05:15] And was created over 18 months ago [05:15] Yup. Delete. [05:15] nigelb: It gets awkward when jtv starts arguing with himself [05:16] Hehe [05:16] jtv: So we can fix startpoppy.sh to write a pid file [05:17] Personally, I don't care at all. [05:17] I track it down via ps and shoot it in the face [05:17] Which is wonderful makework but personally I'd like to avoid participating in it. [05:18] Then don't complain when poppy doesn't have a pid file :-P [05:20] * nigelb looks for another interesting bug to start work on. [05:21] StevenK: I'm not complaining that poppy doesn't have a pidfile, I'm trying to clean up one bit of the mess that dogfood has become by people personally not caring at all. [05:21] wow, mega disrupted day. where did it go. [05:24] How do I create a private branch again? Registering a branch before push only seems to let me do a +junk branch [05:25] stub: code.launchpad.net/launchpad [05:26] OTOH 2 branches landed [05:26] stub: we should do a catchup call; we missed last week (but not right now) [05:27] k [05:27] Think we missed the previous week too :-) [05:27] what's the recommended way to update devel? [05:27] rocketful-get or just doign bzr pull inside it? [05:29] I've done rocketful-get, but I don't know if its "The Right Way (tm)" [05:29] rocketfuel-get is a good way to make sure sourcecode/download-cache/devel are up to date. [05:30] hmm, wonder rif I can get two branches landed on same day. *tries* [05:31] stub: we got the week before ;) [05:32] anyhpw [05:32] -> groceries [05:35] StevenK: the dogfood:ubuntu did do the trick after all, thanks! [05:35] I thought the "/ubuntu" in my config would obviate that. [05:35] BTW poppy has a pidfile now, and startpoppy.sh has a sibling stoppoppy.sh. [05:35] all information related to user profile like timezone, go into registry, right? [05:48] What changed with gpgme recently? [05:49] We're using an egg instead of a branch. [05:49] Which is good. [05:50] No. [05:50] It's just better than a branch. [05:53] Make is failing until devel is merged in [05:53] poolie: You no longer need to retarget to null. [05:54] poolie: Expand the bugtask row and retarget to Ubuntu directly. [05:54] (gpgme is gone from sourcecode, but buildout won't build the egg until versions.cfg etc. has the update) [05:56] We don't test logout() anywhere. Fails when it attempts to redirect through codehosting to clear any secure codehosting auth cookie, which isn't running under the test suite. [05:58] ok, small world. I just realized who the author of pytz is. [05:59] stub: It's not tested end-to-end, no. [05:59] stub: That would be difficult, as I don't think testopenid supports logout. [06:00] Which would have been the next roadblock :-P [06:00] (it doesn't support login) [06:01] It doesn't? I thought it was the same as we use running a local dev instance. [06:01] (And logout fails on my local dev instance because... no codehosting.) [06:02] The redirect path is launchpad->codebrowse->OP [06:02] stub: There's a special make run for codehosting on .dev [06:02] I don't think testopenid.dev exposes a logout interface. [06:02] Or is it just my setup that doesn't have a working bazaar.launchpad.net? [06:02] Because it doesn't have authentication. [06:02] run_codehosting or something [06:03] Well, doesn't cache authentication. [06:03] So it would be a noop [06:03] we should move auth cookie maintenance to a microservice anway [06:03] It's SSO all the way down? [06:03] * stub has a single line fix and no way to test it [06:04] stub: Oh? [06:04] wgrant: well, one possibility is to talk directly to sso; but that won't work with being openid consumer unless we teach sso about that too [06:04] I'll try getting a codehosting setup running locally [06:04] So the hardcoded redirect path works [06:04] wgrant: another possibility is a microservice (xmlrpc on the main appservers initially) which maintains session cookies [06:05] StevenK, wgrant: any objections to a publish-ftpmaster run on dogfood? [06:06] lifeless: Oh, you mean for codebrowse? [06:06] lifeless: Not sure how that solves the problem, which is we need to clear cookies on a number of domains that are not always available. [06:06] stub: 'make run_codehosting' rather than 'make run' [06:06] jtv: None. [06:06] stub: it removes the cookies from those domains [06:06] jtv: Only that it will take EONS [06:06] StevenK: all the more reason to get started as soon as possible! [06:07] lifeless: How does that work without requiring the authentication being treated as a web bug and blocked? [06:07] Also, this run shouldn't take as long as normal ones do. [06:07] stub: let me check an assumption: these are all LP domains right ? *.launchpad.net ? [06:08] lifeless: Not quite. We go all the way through to SSO. [06:08] oh, yes. So no idea why we need to bounce through codehosting. [06:08] Well, unless you count login.launchpad.net as *.launchpad.net, in which case get out. [06:08] wgrant: I do [06:08] Die. [06:08] anyhow [06:08] Don't mess with SSO. [06:08] login.launchpad.net is hopefully not long for this world. [06:08] redirecting to SSO isn't what I was mumbling about. [06:09] We should treat it opaquely. [06:09] it was codebrowser + the wikis [06:09] which we only slightly log out of at the moment [06:09] The wikis aren't part of LP yet.... I'm not sure we should go near them. [06:09] anyhow, the real key point is we don't need to clear the cookie [06:09] we need to invalidate the session [06:10] Yes. [06:10] which is really quite different [06:10] We need to rewrite all our session management for everything. [06:10] Because it's wrong and inconsistent and hard to manager. [06:10] s/r\././ [06:10] Yes. [06:10] I can't type. [06:11] aiieee [06:11] that paged in a phil collins song [06:11] logout is still broken running run_codehosting [06:11] Hah. [06:11] stub: What happens in that situation? [06:11] I don't seem to have bazaar.launchpad.dev [06:11] StevenK: 503 [06:11] check your hosts [06:12] StevenK: you don't get logged out [06:12] Well, you only get logged out of LP. [06:12] Rather than LP+codebrowse+SSO [06:12] % grep bazaar.launchpad.dev /etc/hosts [06:12] 127.0.0.99 bazaar.launchpad.dev [06:12] which is still horribly partial [06:12] do we invalidate api tokens too [06:13] ? [06:13] Not sure if serious. === jtv is now known as jtv-eat [06:15] me? I am, logging out really should Log You Out. [06:16] What if I want my many cronscripts to not need resetting every time I want to log out on some machine somewhere? [06:16] provide an advanced partial logout [06:16] sane defaults and all that [06:16] your many cronscripts case is not the common case [06:16] So logging out on one machine should log you out of *all* machines? [06:16] There should be a way to revoke sessions and tokens, sure. [06:16] But having that be the default is insane. [06:17] INF [20110822-13:16:27.297] [47438788081408] loggerhead: Processed err https://bazaar.launchpad.dev/%2Blogout?next_to=https%3A%2F%2Ftestopenid.dev%2F%2Blogout [0.001 seconds]: ['TypeError: oops_start_response() takes exactly 2 arguments (3 given)\n'] [06:17] lifeless! [06:17] stub: thank you! [06:17] lifeless: Does any other webapp do the s/Log Out/KILL EVERYTHING/ thing you are suggesting? [06:17] fixing [06:17] I've never seen it. [06:18] wgrant: its recommended by security folk AIUI [06:18] I've seen revoking all sessions and tokens on *password changes* recommended. [06:18] But never on logout. [06:18] wgrant: the distinction between oauth, browser session and machine session is not well understood by casual users [06:18] So we should make the default action blow everything up for advanced users? [06:19] tokens should only be revoked that are being used by the web browser on logout, not the ones that have been setup for use by command line/desktop integration etc. [06:19] Let me just log out... [06:19] Oh fuck now I have to reauthorize 30 scripts on 10 servers. [06:19] wgrant: don't conflate default-setting with UI [06:20] we could fix the 'get /logout' logs you out bug, moving logout to an actual post; supply a confirmation form with everything checked and let you unselect stuff you want kept (with the current session having an 'online this session' option beside it for quick selection. [06:20] Logout isn't logout everywhere - it is logout the web browser. Not logout on your laptop too, and deauthorize all your other clients. [06:20] stub: this is exactly the point, that subtlety is lost on casual users. [06:21] stub: they don't understand that e.g. the 'log in with gmail' button establishes a persistent 3rd party session [06:21] It doesn't [06:21] (ok - replace with twitter oauth argument) [06:22] We should certainly fix the OAuth token view to be a general authentication session management view, and make that obvious and next to the logout button. [06:22] But the browser logout button should log the browser out. [06:22] Not destroy the world. [06:22] from where does things go into the context? [06:22] I don't think the subtleties are lost. Most of what I see is that people expect their tokens to be destroyed on password change, not logout. [06:23] In a view, 'context' is normally the object that the view is on. [06:23] nigelb: ^^ [06:24] stub, lifeless: Facebook doesn't do this, AFAIK. [06:24] And if Facebook doesn't, then I think even non-technical users are probably not too terrible about this concept. [06:24] wgrant: argh. So, how would I add a variable to that? [06:24] ugh [06:24] (replace "Facebook" with "99% of websites on the Internet" and it will still fit) [06:24] its wsgi oops thats broken [06:24] fixing [06:24] who knew... http://www.python.org/dev/peps/pep-0333/#the-start-response-callable [06:25] wgrant: No, which surprises people when they change their twitter password and linkedin is still connecting. [06:25] nigelb: Find the class, add an attribute to it. [06:25] wgrant: But that is password change [06:25] stub: Yes. [06:25] wgrant: the class I found from configure.zcml, but I can't find the other attributes defined there, which confuses me. [06:26] stub: I think defaulting to reset stuff on password change may be permissible, as long as it's very obvious what is about to happen. [06:26] But on logout is just insane. [06:27] wgrant: yes, and reset tokens on password auth is what I've seen recommended. I don't think I've seen it recommended for logout either (I thought initially lifeless meant the tokens being used by the javascript api, but I'm not sure they actually exist?) [06:27] stub: JS just uses the cookie for now. [06:27] We should certainly make it easy to do. [06:28] At present there's no way to revoke a remote machine's session. [06:28] Unless you are stub. [06:28] urg [06:29] s/machine/browser/ [06:30] thought you meant an oauth token there :) [06:31] No, those have had a view to manage them since day 1 :) [06:36] Anybody up for a rather trivial review? https://code.launchpad.net/~wgrant/launchpad/bug-830803/+merge/72378 [06:36] And https://code.launchpad.net/~wgrant/launchpad/bug-830849/+merge/72379 is a little bigger. [06:39] wgrant: r=me for the first. [06:40] wgrant: For the second, r=me, if there are already tests for _syncSourcePackages [06:40] stub: http://paste.ubuntu.com/672219/ should fix it for you [06:40] stub: I'm working on a test now [06:41] stub: (thats to python-oops-wsgi, easiest way is for you to checkout the trunk, set develop= . ../path/to/that/checkout and run bin/buildout again [06:41] Nah, its easier to do something else until it winds its way through ;) [06:42] StevenK: Thanks. It is tested somewhere (I ran into it some time in the last 15 branches), but I will add direct tests on transitionToTarget. [06:43] wgrant: In that branch itself? [06:43] StevenK: Yes. [06:43] StevenK: review? https://code.launchpad.net/~nigelbabu/launchpad/4595-upgrade-bug-linking/+merge/71575 :D [06:44] nigelb: You know, stub is OCR today :-P [06:45] Is there a page saying who's reviewing when? [06:45] https://dev.launchpad.net/ReviewerSchedule [06:46] ah [06:46] stub: could you review the brach above? [06:46] :) [06:46] yo [06:50] nigelb: Not sure I approve of calling a private bug a bug with an invalid link - seems a separate case to test to me. [06:50] Hahahah [06:50] He was doing that, but then lifeless said they were the same case :) [06:50] That ^ [06:51] Also, its similar to what happens in the UI. [06:51] I'm open to changing the text to make it clearer. [06:53] I think it is better to be explicit. Saves fallout if we linkify private bugs (which we can do if it is client side). [06:53] So, say "Bug 123 is either invalid or not visible to you" [06:53] <_mup_> Bug #123: There's no direct way to see the project info when translating it < https://launchpad.net/bugs/123 > [06:53]