[00:02] lifeless: Our requests don't have an IAnnotations adapter? === Ursinha is now known as Ursinha-afk [00:31] lifeless: there is a .annotations attribute for that purpose [00:31] I hope it is available as an IAnnotations adapter, but I don't remember if it is [00:32] But, obviously, the direct access is pretty clear [00:36] wgrant: well, webapp/adapter uses thread locals rather than looking up the request and using that [00:36] gary_poster: thanks [00:37] gary_poster: so its .annotations.thing ? [00:41] lifeless, .annotations is a dict [00:41] ok; request.annotations[ [00:41] 'thing'] [00:41] yeah [00:41] I don't think I can use it for this work anyway, because webapp.adapter is involved [00:42] but I may do a pass at that to make it better in future. [00:42] wgrant: gary_poster: did you see my question above about timelines ? [00:42] no, looking [00:42] in case you didn't, would value both your responses to : [00:42] If I said object X is "a request timeline", what would you think it does ? [00:42] I would think it would what I called in another context an "object log": [00:43] an iterable of records of things that happened to the object [00:43] in order of when they happened [00:44] where each record is an object or a dict or something else code-friendly [00:44] (as opposed to just a string) [00:44] so, that, for the request [00:44] I'd think it would record events like request start, traversal start, traversal finish, template rendering times, that sort of thing. Possibly SQL too. [00:45] yeah, that's what I'd expect to see in what I described [00:52] ok cool [00:52] lp.services.timeline is born [00:53] last 'boy don't you wish robert knew zope well' question [00:53] is there a way to 'get me the current request please' ? [00:53] lifeless: get_current_browser_request() [00:53] mwhudson: thanks [00:53] lifeless: it's in canonical.launchpad.webapp.mumble [00:53] probably .publisher [00:54] ok [00:54] and for tests, how does one set that ? [00:54] req = ... [00:54] login(ANONYMOUS, req) [00:58] ah sweet [01:02] gary_poster: was that previous code open ? Could we just grab it ? [01:02] lifeless, ZODB based IIRC. I'll look... [01:05] lifeless, not ZODB directly, actually, but I don't think it is quite what you want: http://pypi.python.org/pypi/zc.objectlog FWIW [01:10] yeah, different focus [01:10] this is small enough, I hope, that any duplication of effort will be minimal [01:10] if not, we can merge the common bits later [01:10] thanks [01:11] I'm off to sign back the rental property [01:14] lp.services.timeline is born ?!? [01:14] lifeless: is this what I think it is? [01:18] I don't know, what do you think it is? [01:20] thumper: ok, really gone; back ~ 3 and we can diskuss then [02:18] thumper: turns out I can leech some wifi here :P [02:18] :) === Ursinha-afk is now known as Ursinha [02:23] thumper: so you were asking what lp.services.timeline is [02:24] I was [02:24] +"""lp.services.timeline provides a timeline for varied actions. [02:24] + [02:24] +This is used as part of determining where time goes in a request. [02:24] + [02:24] +NOTE that it is not LP's timeline-view for products, though they are similar in [02:24] +intent and concept (If a better name presents itself, this package may be [02:24] +renamed). [02:24] + [02:24] +Because this part of lp.services, packages in this namespace can only use [02:24] +general LAZR or library functionality. [02:24] +""" [02:26] :( [02:26] what were you thinking it was? [02:27] A project timeline, which everybody has wanted for approximately ever? [02:27] Ah, I see that's covered in the docstring. Heh. [02:28] grep for timeline in the source; there is a timeline thing already [02:28] product timeline that is [02:28] Er. [02:28] I've never seen one. [02:29] not an aggregated timeline of interesting events [02:29] (which every other hosting site has) [02:29] I have a half-baked plan for project timeliens [02:29] I really should put it back in the oven some more [02:29] I'd love timelines [02:30] for objects [02:30] rss feeds interact with this [02:30] * thumper nods [02:30] everything in a timeline should be in the feed [02:30] and vice verca [02:30] IMNSHO [02:30] agreed [02:30] Yep. [02:30] lifeless: perhaps we (two) should have a sprint on this? [02:30] lifeless: you could come down [02:30] But feature development doesn't seem to happen much these days. [02:30] or you could come up :) [02:30] lifeless: or we could just remote sprint [02:31] thumper: I'd be delighted to help you scratch that itch [02:31] lifeless: but actually dedicate time to it [02:31] lifeless: it'd have to be after francis is back [02:31] when we have RFWTAD I'll have a slot open in kanbam [02:32] :( [02:32] just noticed I have a crack in my laptop keyboard [02:32] probably from carrying it around badly [02:36] :< === Ursinha is now known as Ursinha-afk === Ursinha-afk is now known as Ursinha === Ursinha is now known as Ursinha-afk [02:52] is there any way to do a reverse view lookup? [02:52] from a name to get the layer? [02:52] 'the', not 'a' ? [02:56] lifeless: a view may or may not have a layer defined [02:56] lifeless: I'm after a way to simplify some of the tests [02:56] lifeless: so instead of having to specify the rootsite of 'code' for the +recipe view [02:57] we know that the +recipe view is only ever on the 'code' site [02:57] so we shouldn't have to pass it in [02:57] makes sense to me [03:04] thumper: basically you want to know 'for which X will queryMultiAdapter((context_object, X), name='+recipe') not return None' ? [03:05] mwhudson: yeah [03:09] thumper: doesn't look easy at all [03:09] mwhudson: no === Ursinha-afk is now known as Ursinha === Ursinha is now known as Jorjao === Jorjao is now known as Ursinha [05:32] this stuff isn't half crufty [05:33] also, I hate the import fascist with a vengeance [05:33] lifeless: Well, *duh*, it's a fascist. [05:33] just saying === StevenK changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 2 of 10.09 | Performance Tuesday! | PQM is OPEN | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews [05:34] also [05:34] its a bit annoying that set_request_started *doesn't take a request* [05:34] anyhoo [05:34] I think I'll have a timeline happy by EOD [05:36] stub: thanks for fixing the ppr [05:36] stub: did you have a chance to look at the dba tagged bug ? [05:37] Didn't fix a thing. Just devpad being its usual reliable self I guess. [05:37] the test suite breaks -hard- when set_request_timeout is naffed [05:37] stub: heh, well you can take the credit :) [05:37] Always do [05:48] WTF [05:48] Why am I in the db_lp blamelist? [05:49] I haven't had anything landed for quite a while. [05:53] stub: so [05:54] stub: dba bugs - did you have a chance, or not ? === Ursinha is now known as Ursinha-afk [05:54] did one, can't remember the other [05:55] https://bugs.edge.launchpad.net/launchpad-project/+bugs?field.tag=dba [05:55] both pending your input [05:57] wgrant: oh didn't we tell you about that patch to buildbot? lp:~spm/buildbot-configs/always-blame-wgrant we figured you'd understand [05:58] * spm runs away on the school run; so as to allow a colder dish of revenge to be savoured by M. Grant. [06:00] Heh. [06:05] matsubara-afk: hi [06:11] * thumper EODs for now, prolly back later for more chats :) [06:14] thumper: ciao, we should have a weekly when you're back [06:40] thumper: bug 615647 needs your input [06:40] <_mup_> Bug #615647: BranchMergeProposal:+index timeout [06:40] stub: did the OOPS on that one help at all ? [06:53] it demonstrates that the normally fast query can block. I suspect branch scanner inserting 10k or more rows to that table in a big transaction. [06:53] ok [06:53] is there something we can do to gather data about the cause of blocks like that ? [06:53] It also shows the master db is being used, but for this page that is silly [06:54] (I thought MVCC is meant to help) [06:54] Not yet. [06:54] MVCC does help. We are maintaining indexes on half a billion rows remember. [06:54] true [06:55] I wasn't meaning to be critical -- I just thought that the design meant readers weren't blocked [06:55] So PG 8.4 will help. Code team has eventual plans to drop this table entirely. There is a branch that has been lost dealing with dropping the id column (narrower table will perform better and speed replica and staging rebuilds noticeably and save several gigabytes of disk) [06:57] The design does mean readers should not be blocked. To be honest, I don't understand why it could be blocking for that long. [06:57] should we escalate to our support mob ? [06:57] They might be able to tell us why. Not sure if that helps us though. [06:58] Normal pg mailing lists would give the same or better answer. [06:58] Would depend on the answer I think. [07:00] I think we should contact someone ;) [07:01] spm: yes, i can see the issue there; the respondant isn't all charm either. [07:01] lifeless: oh yes. I wasn't going there. A sledgehammer has *two* faces after all. :-) [07:02] So, seriously, what's with that db_lp failure? [07:03] I killed the ec2 instances. i suspect they should be focibly restarted. [07:03] Yes, but why am I in it!? [07:04] because you have a godlike demeanor [07:04] ... [07:05] ^W^Wchildlike demeanor ? [07:05] wgrant: seriously, noone knows. [07:05] Start with #postgresql [07:05] wgrant: is it worth digging ? [07:05] lifeless: Probably not. [07:05] stub: cool, thanks [07:06] Just wondering if there was any obvious reason (eg. it started two weeks ago and hung). [07:06] It concerns me when the CI system does inexplicable things. [07:06] I don't think db_lp and lp have been building for a while ... [07:06] Since lucid_{db_,}lp is the new hotness [07:16] StevenK: I repeat, I checked [07:16] StevenK: they are both meant to build until we migrate completely. [07:16] bbiab, foodingk. [07:23] 2010-08-08 21:19:20 BST LOG: process 21247 still waiting for ShareLock on transaction 1884213882 after 1000.106 ms [07:24] Different problem, but some insight into the xmlrpc slow stuff [07:24] ouch [07:50] Morning! ;) [07:50] Does anybody have an idea how an error like this can happen on an EC2 test run? [07:51] http://paste.ubuntu.com/482770/ [07:51] jtv: ^ that's on the recife branch after merging db-stable. [07:51] henninge: looking... [07:53] henninge: on ec2, that's strange. [07:53] stub: any idea why ec2 might give the "you need to run make schema" error? ^^ [07:54] maybe need to rebuild the sampledata? [07:57] I don't know why that would happen [07:58] Perhaps a badly named db patch, which is picked up by the has-this-patch-been-applied checks but not picked up by the apply-patches script. [07:58] Perhaps something broke the patch, and we ignore that failure (e.g. because we know that this check will cover it)? [07:59] Look for oddities in the patch filename, such as _ instead of -, not enough digits, typo in 'patch' or 'sql'... [08:00] Despite that message the test suite runs without fail or error. [08:00] stub: it says patch-2207-96-0.sql has been applied to the db but is not in the source tree… anything weird about that name? [08:01] Yes, it is old. The new baseline landed and patches are now 2208, not 2207 [08:02] Is LaunchpadDatabaseRevision in the sampledata? [08:02] dunno [08:03] stub: yup, it is [08:03] stub: So we need to rename the patch to 2208-96 ? [08:03] So chances are the patch was removed from the directory but not from the sampledata. [08:04] henninge: is that our Recife patch by any chance? [08:04] In which case we may need a new patch number for the 2008 series… [08:04] henninge: Yes - patches numbered 2207 will explode. I can give you a new number of you had an official one given out. [08:04] jtv: yes, of course. ;) [08:05] stub: I don't think it was an official number. [08:06] The Translations team frantically digs for the DB review... [08:09] henninge: my version of the recife branch doesn't have that patch any more. :( :( :( [08:12] henninge: hmm… looks like our patch was numbered 99, not 96 [08:13] So what was in 96 [08:13] ? [08:13] Was there ever a patch-2207-96-0.sql? Google says no. [08:13] No idea [08:14] Wow, the mystery of the patch that never was ... [08:14] I just found one hit on google, but I'm fairly sure it's not supposed to redirect to 01proxy.com [08:14] DON'T CLICK! [08:14] ;) [08:14] Too late [08:15] It's actually a hit on 01proxy. :( [08:16] Google's cached version of the 01proxy.com page shows an MP by Curtis, rejected by himself... MP 22810. [08:16] bug 538024 [08:16] <_mup_> Bug #538024: No way of saying "This project is not packaged" [08:16] I need a tip/hand with 'requests' in 'scripts' [08:16] Patch description: "Added date_last_packaging_check column to Product." [08:16] specifically :- [08:16] I have some code [08:16] it keys off of request as its context [08:17] but scripts which haven't setup a 'current_browser_request' try to use it. [08:17] e.g. checkwatches. [08:17] is calling 'login(ANONYMOUS, ScriptRequest())' there sane ? [08:17] or should I just make my code (which is the sql gathering oops helper) silently do-nothing ? [08:18] (its defined as doing nothing in that case anyway) [08:18] mwhudson: ^ I hear you've done a bit of out-of-appserver-stuff, care to comment ? [08:19] Why do I keep seeing this when running make clean? [08:19] rm: Entfernen von „/var/tmp/bazaar.launchpad.dev/rewrite.log“ nicht möglich: Permission denied [08:19] (No, I am not talking about the German text for "removal not possible" ... ;) [08:19] ls -l /var/tmp/bazaar.launchpad.dev/rewrite.log ? [08:19] because it gets created as root by Apache [08:20] it's seriously annoying [08:20] lifeless: I already sudo deleted it ... [08:20] not to mention having the rewrite process running all the time (which I disabled) [08:21] bigjools: Thanks, it's not on my end, then. [08:21] henninge: "export LANG=C ; make clean" should sort out the german text problem? <== not helping :-P [08:21] Hi spm! ;-) [08:21] heya! [08:21] LANG=queens_en.utf8 [08:22] good morning [08:22] bigjools: that'd be en_QUEENS.UTF-8 and it'd probably be taken by Queens, NY [08:22] hi adeuring [08:23] jtv: impossible! [08:23] hi jtv! [08:23] bigjools: not at all. They invented democracy and English, so why shouldn't they have grabbed this one as well? [08:25] so, any comment on my login query ? [08:25] it feels wrong to use lp.testing.login() in a script. [08:25] jtv: ha :) [08:26] lifeless: no idea on this end, sorry [08:26] thanks [08:26] (just saying: you're not being ignored in favour of jokes) [08:26] jtv: hey, I havne't reviewed your thing yet. [08:26] lifeless: careful—Quotes is just a few clicks awa [08:26] y [08:26] jtv: please get a general review of it done and land it; I'll look at it when I get a cycle [08:27] lifeless: ok. Not much there, it's really tiny. [08:35] good morning [08:35] lifeless: why are you logging in for a script? [08:35] jml: hi [08:35] hi jml, hi thumper [08:36] jtv: do you know the recursive sql syntax for postgresql? [08:36] jtv: or put it another way, could you write a correct recursive query? [08:36] jtv: mine didn't work [08:36] thumper: haven't tried yet [08:36] jtv: ack [08:36] Still too intimidated by the SQL Mandelbrot [08:37] All I remember is the "with" is basically a pipe, and you can feed the outcome of the query back into that pipe as it were [08:37] thumper: what kind of problem are you trying to solve? Transitive closure-type stuff? [08:38] jtv: trying a recursive query to identify if a revision is in a branch based on the revision and revision parent tables avoiding the branch revision table [08:39] I'd love to be able to test the query on the staging db [08:39] jtv: part of the problem though is that the tree we are traversing is likely to be deep and narrow === almaisan-away is now known as al-maisan [08:40] thumper: this was actually one of the test questions I was asked as part of the hiring process… guy named Pool. [08:40] Wonder what made him think of this problem. :) [08:41] thumper: we should have a participation in place [08:41] eh? [08:41] thumper: for security proxies etc to work [08:41] thumper: and for things keyed off of that to work too [08:41] lifeless: ah, EOLDCONTEXT [08:43] are we in testfix mode? [08:43] thumper: I've refactored get_request_statements to not use a thread local variable [08:43] thumper: and to store much more than sql [08:43] like email, rabbit, memcache, bzr times [08:44] but, sadly, I now need to learn about our scripts stuff [08:45] thumper, I guess so, since the last build "failed" [08:46] jml: ah, you know about this stuff; see my question above [08:46] lifeless, which one? [08:47] 19:16 < lifeless> I need a tip/hand with 'requests' in 'scripts' [08:47] ... [08:48] lifeless, in answer to your question, yes, I think using ScriptRequest() there is sane [08:48] hmm [08:48] jml: so, setupInteraction(ANONYMOUS, participation=ScriptRequest()) ? [08:49] or will we need a celebrity or some such to represent the scripts (do we have one already) ? [08:49] lifeless, I think so. No celeb. [08:49] Don't scripts mostly run with PemissiveSecurityPolicy? [08:49] it's been so long since I needed it. [08:49] Soyuz ones do, at least. [08:49] wgrant: I've not heard of that [08:49] wgrant, they (almost?) all do [08:50] lifeless, ahh, right. [08:50] some more context for you then! [08:50] but if it means 'ANONYMOUS can write' then I'm happy [08:50] they also load less zcml [08:50] in a half-hearted effort to have them run faster [08:50] Do they really? [08:50] * wgrant checks. [08:50] wgrant, they used to [08:50] It certainly doesn't feel like it. [08:51] wgrant, I should say "a subset of our zcml" to hedge my bets. [08:51] Heh. [08:52] initZopeless is key here, iirc. [08:53] Well, sort of. [08:53] execute_zcml_from_scripts is a separate thing. [08:53] But they are usual called alongside each other. [08:53] ahh right. [08:53] s/from/for/ [08:53] it floods back like a wave of mutilation. [08:53] initZopeless does the transaction setup. [08:54] execute_zcml_for_scripts does ZCML and security and stuff. [08:54] and exec_zcml_for_scripts sets up the interaction [08:54] Scripts call both. [08:54] Yep. [08:54] # This is a convenient hack to set up a zope interaction, before we get [08:54] # the proper API for having a principal / user running in scripts. [08:54] # The script will have full permissions because of the [08:54] # PermissiveSecurityPolicy set up in script.zcml. [08:54] setupInteractionByEmail(ANONYMOUS) [08:55] so [08:55] lifeless: Scripts shouldn't need anybody logged in. [08:55] is it, in principle, as simple as adding ScriptRequest() to that ? [08:55] In PermissiveSecurityPolicy, all permissions are held at all times. [08:55] This, confusingly, does not mean that all operations are possible. [08:55] should is a strong term [08:55] lifeless, not sure. It ought to be. [08:56] lifeless, try it? [08:56] jml: I will. [08:56] lifeless, "should" is an ambiguous term. [08:56] jml: what file was that ? [08:56] lifeless, c.l.scripts.__init__ [08:58] jml: why do we use tac files instead of just running the reactor from a .py? [08:58] bigjools, because we get daemonization and logging and a bunch of other stuff from twistd for free [08:58] ok [08:59] jml: its a bit annoying that that isn't a two-liner to get from a py file [08:59] lifeless, yes. known bug. === jamesh_ is now known as jamesh [08:59] it's *also* a bit annoying that there are tac files and there are "twistd plugins", and that the latter are somewhat nicer to use but harder to write. [09:00] agreed [09:00] Hmm. [09:01] I was about to say that it was unfortunate that no-one pays me to hack on Twisted full time. [09:01] hah [09:01] AttributeError: 'ScriptRequest' object has no attribute 'interaction' [09:01] But I've never really seriously pursued it as an option. [09:01] Hmm. [09:02] I can't remember if the standard request object is a participation or whether it gets adapted to one. [09:03] I'm digging [09:03] the root of this is me wanting to write clean code [09:03] tsk tsk. [09:04] and adapter._local being a) fugly and b) not in lp.services [09:09] wgrant: going back to what scripts need. [09:10] wgrant: I think its reasonable to say that scripts are always acting on *behalf* of someone modelled in the systme. [09:10] wgrant: be it a celebrity or user [09:10] lifeless: Probably, yes. But that's not how they're implemented. [09:11] wgrant: being implemented differently gives us two environments to write code for, without justification for that split. [09:11] wgrant: this is, I argue, harder than it needs to be. So we should change it. [09:11] Oh yes, and it results in ugly ZCML and probably security holes too. [09:11] however, the trivial didn't work, so I'm going to workaround, bug is filed, putting details in it. [09:13] an interesting tidbit of information... [09:13] approximately half of the 600 million branch revision rows are for launchpad branches :) [09:15] jml: wgrant: I would appreciate comments on https://bugs.edge.launchpad.net/launchpad-foundations/+bug/623199 [09:15] <_mup_> Bug #623199: scripts do not establish valid zope partiticipations [09:16] thumper: How hard would it be to remove the table itself, and not just the primary key? [09:16] :) [09:16] wgrant: there are half a dozen use-cases for it [09:16] thumper: I mean, it's all redundant... but I guess postgres might not be excellent at dealing with that sort of data. [09:16] which I'd like to replace [09:16] well [09:16] for one, it needs to be migrated from SQLBase [09:16] if it hasn't already [09:16] lifeless: it needs to be stabbed in the neck [09:17] lifeless: No, it needs to be deleted. [09:17] wgrant: hi five [09:17] 'to delete the id column you cannot use SQLBase' [09:17] It's moving, kill it! [09:17] or high five [09:17] lifeless: I want to delete the *table*. [09:17] I was answering your fairly specific question [09:17] Not just the column. [09:17] someones [09:17] wgrant: jam had some interesting ideas to make it better [09:17] StevenK: More "It's redundant, huge and slow, kill it!" [09:18] wgrant: preprocessing is pretty standard for graph problem domains [09:18] this particular preprocessing layout isn't ideal. [09:18] but running from the adjacency list is also going to be terrible. [09:19] if you're interested in tackling this, RMQ transform papers are a good place to start to get ideas. [09:19] * thumper EODs again [09:19] something like the graph db stuff twitter has opened source might be nice too [09:19] Morning [09:20] mrevell, good morning. [09:20] lifeless, what about storing it all in one big 2a repo? [09:20] wgrant: Suprised you haven't commented on my latest branch, Stalker! [09:21] lifeless, hey, you know what would help me write cleaner code? [09:21] wgrant: And, "It's moving, kill it" is a movie reference, I just can't remember which. [09:21] lifeless, usability improvements to testrepository :P [09:23] StevenK: Uni workload is making my stalking less effective, sadly. [09:23] A GINA BRANCH!? [09:24] What is this? [09:24] Nobody works on gina. [09:25] jml: yes, they are really high up [09:27] wgrant: I just proved you wrong :-P [09:29] :( [09:29] We delete the BranchRevision.id column because it is unnecessary and dropping the entire table is a larger, unscheduled and not ready to code task. [09:30] Think of it like getting 10% more effective ram on the master database. [09:32] Well, maybe not that much but quite noticible. [09:32] noticeable [09:43] mpt: hey [09:43] hi lifeless [09:43] mpt: so, have you heard anything negative about the search changes we did? [09:43] lifeless, nothing at all, negative or positive :-) [09:43] well, no noose is good noose [09:43] Search was bad... it's now very slightly worse. [09:45] wgrant: really ? [09:45] wgrant: which search is worse ? [09:46] Dupe search. [09:46] you're finding it noticable less powerful ? [09:46] or you've seen folk comment on it ? === jtv1 is now known as jtv [09:47] I've found a couple of cases where it wasn't really optimal. [09:47] Specifics I cannot currently recall. [09:47] ok [09:47] and that you felt sure the other behaviour would have done bette r? [09:52] stub: I'll look at it with my new fella [09:52] Ta. There was work done at the Epic. jtv will recall the details. I don't recall who was actually doing the work and how far it got. [09:53] huh? [09:53] jtv: Dropping BranchRevision.id [09:53] ah [09:53] I left it in Jon's capable hands, pushing him from time to time to bloody get on with it. [09:53] As I recall, we: [09:53] * Stormified the class. [09:53] jtv: really? did it land? [09:53] The stormification did. [09:54] jtv: but not the id column removal? [09:54] Then, we removed a bunch of references to the id. Not sure whether that landed. [09:54] jtv: thanks, I'll chase jam in the morning [09:54] And I prepared a branch for him to work off of that made it use (branch, revision) as the primary key. [09:55] Is "bloody" a documented permitted splitting of infinitives? [10:00] Fowler's article on the subject is worth a read. [10:01] to go bloody? or to bloody go? valid, and very different :) [10:01] To bloody go where no vampire has gone before! [10:02] stop bloody wasting time [10:04] to boldly split infinitives that no man had split before [10:04] * bigjools eats shoots and leaves [10:13] noodles775: argh—what script produces the BuildFarmJobs again? [10:13] or maybe bigjools knows === adeuring1 is now known as adeuring [10:14] jtv: No script produces BuildFarmJobs on their own? [10:15] jtv: what do you want to do? [10:15] When BinaryPackageBuilds are created, BFJs are also created... [10:15] * noodles775 thinks that's a better question. [10:15] Okay, when are BinaryPackageBuilds created? [10:15] jtv: what do you want to do? [10:16] I want to verify that pushing an intltool-enabled branch that's set up to generate translation templates is really generating TTBJs [10:16] ok [10:16] The scanner creates TTBJs. [10:16] wgrant: thanks [10:16] So 'make sync_branches' will do it locally. [10:17] This is on staging… it's one of those changes that need verification on each environment we deploy them in. [10:17] (also, TTBJs aren't BFJs yet -- but I hope you're going to fix that soon?) [10:17] * bigjools hopes so too [10:17] Yes yes yes, it's on our shortlist! [10:20] stub: I'm trying to get dogfood running and it's bitching about patch-2207-00-0.sql being in the DB but not in the tree. HALP. [10:20] bigjools: I suspect you have patch-2207-* files in database/schema [10:26] yes :/ [10:27] thanks [10:37] well, I'm going to halt(); this fallout is annoying :(. [10:39] so gnight. [10:59] :( [10:59] My branch to eliminate BuildBase is massively oversized. [11:04] Too much code to move around. [11:11] wgrant, by oversized, do you mean "could be split up" or do you mean "more than 800 lines"? [11:12] jml: The latter. [11:12] It could possibly be split, but it's mostly just moving content from a couple of files to a couple of other files. [11:12] wgrant, I'm happy to do the review [11:13] although maybe not before you sleep. [11:13] It's not exactly pretty yet, but I figure I'll get it all moved across before cleaning up. [11:13] I'll propose the review shortly. [11:37] henninge: any luck with the recife branch? [11:38] oh, you missed that, I guess... [11:38] yes, I was out for lunch [11:38] jtv: It's landed. [11:38] thanks henninge—I'll land my branches [11:38] jtv: .. and you were disconnected [11:49] jml: https://code.edge.launchpad.net/~wgrant/launchpad/buildbase-must-die/+merge/33505, if you have time. [11:50] wgrant, sure. I'll do that now, in fact. [11:54] Ah, I feel less bad with that 1800-line rootsite change having just landed :) [12:00] something I'm not going to ask you to change, but feel is worth mentioning: [12:00] > +            build.dependencies = unicode(slave_status.get('dependencies')) [12:00] ^^^ that is a point of fragility. [12:00] Indeed it is. [12:00] I've noticed that many times, but was never sure how to fix it. [12:00] Since it is really a bytestring. === al-maisan is now known as almaisan-away [12:02] Morning, all. [12:02] hi deryck [12:04] wgrant, I guess for a start I would spell it as slave_status.get('dependencies').decode(), give it an encoding and tell it how to handle encoding failures [12:04] Yay everyone, staging's back! [12:06] jml: Well, yes, but the encoding is probably just ASCII, and I don't really know how best to handle failures here. [12:06] So it might as well just crash ;) [12:09] wgrant, perhaps. depends a lot on how helpful Python's exception is. [12:12] _handle_status_OK is... interesting. [12:12] Um, yes. [12:12] It's pretty awesome. [12:12] It's also largely untested :) [12:13] which planet's definition of awesome? [12:14] I meant "interesting" in the "May you live in interesting times" sense. [12:16] Soyuz brings quite enough interestingness, TYVM. [12:16] Hmm. LP looks nice in UbuntuBeta. [12:17] * bigjools orders another kilo of coffee from HasBean [12:18] wgrant, ooh, has it landed? [12:18] jml: Apparently. [12:18] Since launchpad.dev now uses it. [12:19] Although it makes the badly aligned sprites look even more badly aligned. [12:19] :( [12:20] hah, but one revision off from being on edge. [12:20] :( [12:22] wgrant, review done. === almaisan-away is now known as al-maisan [12:24] jml: :( [12:24] But will fix. [12:25] wgrant, thanks. [12:26] wgrant, I always feel a little lousy asking people to change code they've pretty much only moved, and for enforcing the trivial points of our coding standard, so it's very much appreciated. [12:27] I'm all for enforcing the coding standard as much as possible. [12:27] I was just hoping to avoid it in this already massive branch. But the changes you've requested are tiny, so I'll do them. [12:28] ta [12:35] * bigjools is currently making a lp.soyuz.enums [12:36] * wgrant stabs the test suite for not working when a local buildd is already running. [12:36] wgrant, ping me once they're pushed up. I'll submit to ec2 immediately & then go to lunch instanter. [12:36] * bigjools sighs at PackageCopyStatus.CANCELING and PackageCopyStatus.CANCELLED [12:37] jml: Sure. It'll be a few minutes, since I need to do a test build manually. [12:37] np. [12:39] are we enforcing the multiline import style in doctests? [12:39] bigjools, probably yes. [12:39] But they weren't migrated. [12:39] exactly [12:39] wgrant, a sound point. [12:39] I'm not convinced it's a good idea in doctests [12:39] Alternatively, we could just enforce no doctests. [12:40] heh [12:40] I like doctests when they're done properly [12:40] admittedly I don't like much here [12:40] tbh, I don't care as much about the coding standard within doctests. [12:40] since it's not actually code. [12:41] good point [12:42] but I also see little point in being inconsistent [12:42] especially since the documents have no target audience [12:44] mrevell: don't forget to dig up the text & link for the popup balloons… could you email them to me? I'm pretty much at the point where my egg needs a chicken. [12:44] Or vice versa. [12:45] jtv, Yes, no problem. Will do that now. [12:45] thanks :) [12:54] bigjools: I am quite a fan of the new buildd-manager design. [12:54] glad to hear it [12:55] I lost most of my hear, some sweat and some blood making it [12:55] Now we just have to fix updateStatus and others to stop pretending that it's all synchronous. [12:55] yes, I have a sprint planned in September with jml to fix that [12:55] Excellent. [12:56] there will be a method that does a scan and returns a Deferred that fires when that scan is done. [12:56] e [12:56] and there will be much rejoicing among the people. [12:56] also jelmer has a fix in progress that puts build uploads in a queue that we can process outside of b-m [12:56] Yup. [12:56] * jml relogs. [12:56] I wonder how well it will scale after that. [12:56] * bigjools rejoices with jml [12:57] jml: but it's not as simple as that - there's a bunch of synchronous XMLRPC that we need to convert as well :) [12:58] Currently RecordingSlave lies by reporting success for everything :( [12:58] yeah, I have that in my sights [12:58] we can get rid of it and turn the existing BuilderSlave into a twisted implementation [12:59] And that means we have a good reason to kill slave-scanner. [12:59] we don't already? [12:59] IIRC you wanted to keep it. [12:59] It's only like 30 lines, anyway. [13:00] I did, but I rescind that desire [13:00] Yay. [13:00] Since buildd-manager sucks less now? [13:00] since we have more than zero confidence in it now :) [13:00] Heh. === matsubara-afk is now known as matsubara [13:03] bigjools, sure. nothing is ever that simple. but "eyes on the prize" and all that. [13:03] yarp [13:18] Anyone happen to know what to pass into archive.syncSource(to_series=....) in the api ? [13:19] The distroseries name. [13:19] >>> ta.syncSource(from_archive=fa, source_name='subunit', version='0.0.4-4ubuntu4', to_pocket='Release', to_series='jaunty') [13:19] 400 Bad request: subunit 0.0.4-4ubuntu4 in hardy (same version already has published binaries in the destination archive) [13:19] * noodles775 is just reading the api doc. [13:20] maxb: include_binaries=True [13:20] Hm. [13:20] But they're different archives? [13:20] Must already be in the target somehow, anyway. [13:21] * maxb boggles... you're rigjt I was missing include_binaries=True in my interactive example... but my script which failed originally did have include_binaries=True [13:21] So it fails the same way, even with include_binaries=True? [13:21] Yes [13:22] except not any more, because the publisher has run [13:22] Ah. [13:22] The scenario is trying to syncSource the same source+binaries that is published in multiple series in one archive, into the same multiple series in another archive [13:40] gror [14:10] jml: did you forget your pills today? [14:11] bigjools, no. my IRC proxy was playing up. [14:11] now, some 90 minutes later than intended, I'm off to get lunch. [14:27] jamesh, if you have a minute i'd like you to look at a workaround i wrote for bug 620508 [14:27] <_mup_> Bug #620508: Slicing a ResultSet breaks subsequent len() calls. [14:27] jkakar, if you're available, you could do it instead === Ursinha-afk is now known as Ursinha [15:04] james_w: did you see the second question in my lp-dev email (re. regarding comments per package or per version). (it was in a second email after my fat fingers sent the first prematurely ;) ). [15:05] yes, replying now [15:05] Ta. [15:15] james_w: hi, while you're around, I need to remind you to add AGPL headers, module docstrings and __all__ to your new files :) [15:16] yeah, I sometimes forget, sorry [15:16] your reviewer should have seen that :( [15:17] (see standard_template.py and standard_test_template.py in the lp root folder) [15:17] * bigjools is still not sure about seeing PackagePublishingPocket in registry.interfaces [15:17] It does not belong there [15:18] bigjools, wgrant and I talked about moving it last year, but neither of us did it [15:18] didn't jml put it there? [15:18] mrevell: nag, nag. :) [15:18] yes I did. [15:19] in early 2009, if memory serves. [15:20] lp.code needs some way of referring to things like "karmic-backports" [15:20] I think code depends on Soyuz really. [15:20] Code, even [15:21] that aspect of lp.code (i.e. basic branch mgmt) has nothing to do with archives, build farms, upload servers or anything else. [15:22] pockets are a soyuz thing [15:22] baltix, fedora and debian do not use them. Only Ubuntu and future derivatives can use them [15:22] I would be much more comfortable with code depending on soyuz if soyuz were itself more clearly defined [15:23] what is not clear about it? [15:23] sinzui, well, the fact that code needs to care about pockets at all is a bug. [15:23] yes [15:23] that aspect of lp.code (i.e. basic branch mgmt) has nothing to do with archives, build farms, upload servers or anything else. [15:23] I don't want code to depend on all of that. [15:23] and yet all of that is frequently called "soyuz" [15:23] build farms are not soyuz [15:24] maybe I should put the publishing definitions in archivepublisher [15:25] if there were a Suite class, where would it live? [15:26] soyuz [15:27] but distro would stay in registry? [15:27] * jml thinks [15:28] yes [15:28] soyuz is responsible for setting up and managing suites [15:28] I guess I don't know how distroseries & suites relate outside of an Ubuntu context [15:29] I mean, the current relationship is a distroseries has many suites; a suite has a single distroseries [15:29] bigjools, jml, distroseries appears to be a nexus of circular imports. I do not know how to separate soyuz and registry. I am not sure what comes first, packages, or distros [15:30] jml, sinzui: in the Debian world, suite is arbitrary. Only Ubuntu formalised that into series-pocket. [15:30] bigjools: yeah, I know. [15:32] jml: to refine your statement, a distrseries can have one or many suites [15:32] bigjools: is there any qualitative difference? [15:33] sort of, if there's only one suite then its name is the same as the series [15:33] isn't that an implementation detail? [15:33] I'm trying to figure out what "Target to release" means in relation to suites, if anything. [15:34] what is your context for "target to release" ? [15:34] https://bugs.edge.launchpad.net/launchpad-code/+bug/622290 [15:34] <_mup_> Bug #622290: Color of info box varies after subscribing to a branch [15:34] ugh, sorry. [15:34] https://bugs.edge.launchpad.net/ubuntu/+source/nspluginwrapper/+bug/178038 [15:34] <_mup_> Bug #178038: npviewer.bin crashed with SIGSEGV [15:34] any distro bug. [15:34] ok [15:35] so they don't care about suites [15:35] only the series [15:36] the suite is selected on the basis of archive management [15:36] (in the case of Ubuntu) [15:36] hmm. perhaps it's N:N rather than 1:N, and only 1:N for Ubuntu. [15:37] that wouldn't make sense to me I don't think [15:37] but maybe I'm too used to the status quo [15:38] well, what I really mean is that a suite has a single distro, rather than a distroseries [15:39] but of course that would make the way we currently do branches very tricky. [15:51] I should draw a picture. [15:52] I should make it easier to draw computer pictures [15:54] Or someone should port OmniGraffle to Linux. [15:55] back soon. [16:16] back === Ursinha is now known as Ursinha-lunch [16:19] wondering if I should buy http://www.amazon.co.uk/Fundamentals-Queueing-Theory-Probability-Statistics/dp/047179127X/ref=sr_1_1?ie=UTF8&s=books&qid=1282663085&sr=8-1 === matsubara is now known as matsubara-lunch [16:35] jml: I fell asleep during a queuing theory lecture [16:37] bigjools, what? were you marked on attendance at your university? [16:38] no [16:38] it was just so uninteresting [16:38] hah [16:38] it appears to be rather useful though. [16:38] that's the bloody annoying part of it :) [16:40] also, I have great faith in the ability of most lecturers to make any topic as dull as "Stephen Hawking reads the Yellow Pages" [16:42] jml: you attended that one? How did it end? [16:43] jtv, a surprise impromptu concert by ZZ Top [16:43] Man, I knew there was a surprise ending! [16:48] I'm definitely going next time. === beuno is now known as beuno-lunch [16:52] jml: one of my lecturers has a sex change. That was the most interesting thing about the lecture. [16:52] had* [16:52] bigjools, not during the lecture, I hope. [16:52] he announced it at the end of one, and turned up at the next as a bird [16:53] Goodbye Frank, hello Frances [16:53] I'm sorry, but now I've got a picture of Owl from Winnie the Pooh giving you a lecture on queueing theory [16:54] On that note, 'evening all :) === benji is now known as benji-lunch === matsubara-lunch is now known as matsubara === Ursinha-lunch is now known as Ursinha === beuno-lunch is now known as beuno === benji-lunch is now known as benji [18:45] salgado, quick question. given a request object, how can i determine which oauth token it was signed with? [18:45] i would expect to see that in IOauthSignedRequest, but that's just a marker interface [18:46] leonardr, I think you'll have to poke at the request to get the OAuth header and extract the token ID from there [18:47] IIRC, when we do that to authenticate, we extract the person associated with the token and throw the token away [18:47] so you could also change that code to store the token somewhere along with the principal [18:49] get_oauth_authorization(request) is how you'd go about extracting the OAuth header [18:50] salgado, i don't need the token per see, i just need principal.access_level [18:50] that should do nicely [18:58] g'night all. would really appreciate reviews of my two small outstanding patches. [19:11] i realize it may seem strange for me to ask this question, but has anyone published a field that's writable normally but read-only through the web service? === al-maisan is now known as almaisan-away [19:51] moin [20:00] hi lifeless === lifeless changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 2 of 10.09 | Performance Tuesday! | PQM is OPEN | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in === lifeless changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 2 of 10.09 | PQM is OPEN | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews [20:14] lifeless, it's still Tuesday last time I checked... :) [20:17] rockstar: I can put it back if you like, but its been 24 hours; every has woken up to it like that on their tuesday :) [20:17] rockstar: would you like it back? Are you doing performance? [20:17] lifeless, I'm just bustin' your chops. :) [20:17] 'pow'! [20:18] * beuno stops doing performance [20:18] lifeless, actually, I am doing performance checking on the javascript that I'm writing. [20:18] * rockstar has nice tools for that, so it's easy. [20:39] rockstar: nice [21:01] abentley: did you want to talk more ? [21:01] lifeless, you bet. [21:02] lifeless, https://wiki.canonical.com/Launchpad/NewTaskSystemRequirements [21:15] lifeless, apparently, skype isn't a reliable system. === almaisan-away is now known as al-maisan === Edwin is now known as Guest13428 [21:48] lifeless: did you get an answer to your question? === Guest13428 is now known as EdwinGrubbs [21:50] mwhudson: which one? [21:50] abentley: sorry, was waterinating and chatting with Lynne [21:50] is calling 'login(ANONYMOUS, ScriptRequest())' there sane ? [21:50] or should I just make my code (which is the sql gathering oops helper) silently do-nothing ? [21:50] (its defined as doing nothing in that case anyway) [21:50] mwhudson: ^ I hear you've done a bit of out-of-appserver-stuff, care to comment ? [21:50] oh right [21:50] see my mail to the list [21:50] I got a bit of and answer [21:52] ah ok [21:53] * mwhudson -> email [21:54] morning [21:55] 'morning thumper, mwhudson [22:02] thumper, morning. [22:02] abentley: hi === matsubara is now known as matsubara-afk [22:14] mwhudson: see my changes to adapter.py to see the sort of mess i'm making [22:19] thumper: So, I'm not really sure what to do about the chicken repository. It seems to be broken on the server side now. [22:19] loggerhead hates bzr! [22:19] thumper, rockstar, I need to get going at 5:30. Do you want to have a stand-up now? [22:19] http://bazaar.launchpad.net/~bzr-pqm/bzr/bzr.dev/annotate/head%3A/bzrlib/builtins.py [22:19] thumper: I can confirm that at least some of the other http repositories get fixed by the newer version of bzr-git. [22:19] it won't load that for me [22:19] oh, and NOW it does! [22:19] abentley, it looks like thumper's busy with flacoste-like calls right now. [22:19] beuno: it seems to be doing that for a lot of branches - working after one or two refreshes [22:19] abentley: I'm talking with gary right now [22:20] abentley, I'm happy to chat with you right now, but that might defeat the purpose... [22:20] abentley: if you want to chat with rockstar, he can fill me in later :) [22:20] thumper, isn't rockstar also unavailable later on? [22:20] please make bzr files smaller so loggerhead is happy. kthxbye [22:20] abentley: he's coming back :) [22:20] abentley, yeah, I just have class, and then I'll be back. [22:21] rockstar, okay, so let's chat. [22:21] abentley, cool. [22:24] beuno: in this case they should stop editing it so often! [22:24] beuno: of course, loggerhead shouldn't annotate by default either [22:25] we should contract someone to work on loggerhead [22:25] * beuno ducks [22:25] fantastic idea [22:28] sinzui, I'm lovin the project set-up of each app [22:28] the progress bar not so much, as it may be incomplete for ever, which is a bit awkward [22:28] but it's awesome [22:28] beuno, not forever, only for the next week [22:29] sinzui, aha! I forgot you always have a deck of cards up your sleeve [22:29] beuno, we are landing the changes needed to make the progress bar now. We are landing the new app front pages this week and next,.. [22:30] * beuno will keep his eye on it [22:30] Apps are off unless you tell Lp how it is used. [22:31] I expect a lot of blueprint pages to disappear until users users^h masochists decide they want to enable them [22:32] heh [22:32] still no takers to fix blueprints? [22:32] many volunteers, in the bug comments, but not many landings [22:33] * beuno still remembers the 2 weeks he spent in blueprints and shivers === al-maisan is now known as almaisan-away [22:34] beuno, mtaylor said he wanted to fix blueprints [22:34] brave man [22:35] The notification bugs are at least 2 man-months of work [22:42] sinzui: It's really tempting to hide the option to turn Blueprint on unless people go looking for it. [22:43] wgrant, I think we can accomplish that in the next 7 days [22:43] really [22:43] I want the same option for Answers. [22:43] * rockstar heads to class. [22:43] Because Blueprint looks awful, and it's not clear that it's unmaintained. [22:44] that too will happen [22:44] So it should be hidden. [22:44] Apps will behave like bugs. It is not on unless you choose to enable them. It can be implicit in the case or branches. [22:45] Code should always be available. But the use flag needs to be explicit. [22:46] We want people to tell use when the branch is. It is okay to be a mirror, "just tell us" [22:47] Ah, right. [22:47] beuno: yup. I'm going to get them all fixed [22:47] mtaylor, I assume you are bring a portable army to do that. [22:48] mtaylor, I have "personal heroes" page on my blog for people like you [22:48] sinzui: trying :) [22:48] just exposing api means you do not need to fix a lot of the issue...just build a simple replacement app [22:49] sinzui, the problem with that is that the blueprint api is the scariest part. [22:50] * rockstar heads out for real this time. [22:52] jml: Thanks for landing that. [22:52] mtaylor: hey [22:52] mtaylor: getting started on LP hacking now? [22:53] mtaylor: also the location of the LP epic in January has been set as Dallas, TX [22:53] Dallas again!? [22:53] wgrant: i think bigjools is very happy about this, to judge by fb [22:53] yup [22:53] Heh. [22:53] :( [22:54] mwhudson: I saw that last night, but didn't know what it referred to... [22:54] I may have to contract marburg that week [22:54] sinzui: ? [22:55] sinzui is not compatible with texas [22:55] sinzui: hey. If I want to change some css, is modifying style-3-0.css the right way to go to a first approximation? [22:55] (famously) [22:55] * sinzui does not understand why Canonical does not see the upper or lower parts of North America as legitimate places to hold a sprint [22:55] sinzui: like Mexico? [22:55] sure [22:56] Canada might have a conference facility [22:56] running, back later [22:56] gary_poster, style-3-0.css.in if it is a non-app-specific hack [23:02] thumper: oh cool [23:02] thumper: and no - but _real_soon_now_ [23:03] mwhudson: if I can get some cycles form you on this issue I would love that [23:03] lifeless: i've read all my other mail now :-) [23:03] and am reading yours carefully [23:03] mwhudson: thanks! [23:08] lifeless: gnnaaaaaaaaar i hate this duplication of information across different thread-locals :( [23:08] mwhudson: yes, and I want to reduce it [23:09] the primary lever seems to be 'really make interaction^WIRequest-in-an-interaction the context. I mean really.' [23:09] yeah quite [23:09] if thats wrong headed I'd like to back away quickly [23:09] as discussed, no thread-locals would be nice [23:09] but if we have to have one for now, the interaction seems like it should be it [23:09] if its right headed, I need some clue inserted - or collaborated on - to get over the initial hump [23:10] afk for a few, need to unload more wood [23:11] where have the query counts gone from the bottom of the pages? [23:11] and why is LP even slower than normal today? [23:12] this is insane [23:12] thumper: they're still there for me? [23:12] Query counts are still there for me too. [23:12] they might not be present on pages that don't use the main template, or something like that [23:12] And it was really slow yesterday, but it's not so bad now. [23:13] 41 seconds to approve a code import [23:13] no time out [23:13] mwhudson: this was for a branch page [23:13] I think it didn't get the entire page [23:13] as it was missing everything after the ( in the footer [23:14] thumper: as per rob's mail, maybe sending mail from the webapp is slower for some reason? [23:14] ?!? [23:14] that wouldn't impact loading a page [23:14] unless it was queued for ages [23:14] waiting for a thread [23:17] *shrug* [23:17] just a guess [23:21] back [23:22] its entirely possible that a slow mail dispatch routine stalls a webapp request [23:22] there are two bugs [23:22] one, on foundations, that this doesn't timeout-and-go-boom [23:22] two, that we can't tell where the time is going. [23:25] yes [23:26] * thumper goes to encaffeinate [23:26] third bug [23:26] that these external calls are not equipped with timeouts [23:54] Why karmaactions in the DB? [23:55] That's insane, and makes my bootstrap script fragile :( [23:56] wgrant: expand please :) [23:56] mwhudson: so, I tried using ScriptRequest [23:56] but it doesn't implement IRequest [23:57] subclassing BrowserRequest made it start failing to assign during __init__ :( [23:57] well [23:57] I could use LaunchpadTestRequest in script/__init__.py but it may make some people cry [23:57] lifeless: KarmaActions are basically constants stored in the DB. Code expects them to be there. If one is added to the code and you don't add it to the DB, the code will crash. [23:58] the fact that we're calling get_current_BROWSER_request suggests to me that it's perhaps not the thing to use in non-webapp specific code [23:58] They're rather like celebrities, except that they'd be much better handled by having a dict in the tree somewhere. [23:59] wgrant: enums ? [23:59] lifeless: Or that.