[00:27] mwhudson: but it works \o/ [00:29] jelmer: yeah! [00:51] You know, 2a fetch is pretty slow. [00:52] yes [00:52] Well, a bit worse than that, actually. [00:55] heh [00:55] * mwhudson files a bug [00:56] About? [00:56] 2a fetch being slow [00:56] wgrant: https://bugs.edge.launchpad.net/bzr/+bug/562666 [00:56] Bug #562666: 2a fetch is very cpu intensive [00:56] mwhudson: I have a bzr.log for a checkout of that linux import, if you're interested... it took a little longer than I expected: 4 hours. [00:56] mwhudson: have you profiled it? [00:57] jelmer: no [00:57] wgrant: sigh [00:57] wgrant: how big is the repo in the end? [00:57] mwhudson: ~400MB, it looks like. [00:57] not bad [00:57] git is 312megs it seems [00:58] .bzr in my shared repo is 383MB. === Ursinha is now known as Ursinha-afk [00:58] wgrant: can you do repository/packs and repository/indices as well? [00:58] I am tmepted to try a local branch, but then I might be waiting all day. [00:58] wgrant: it will probably take about an hour [00:58] wgrant@magrathea:~/dev/linux/bzr$ du -sh .bzr/repository/packs [00:58] 319M .bzr/repository/packs [00:58] wgrant@magrathea:~/dev/linux/bzr$ du -sh .bzr/repository/indices/ [00:58] 64M .bzr/repository/indices/ [00:58] wgrant: thanks [01:00] mwhudson: Want bzr.log too? [01:00] It was 'get_parent_map'ing until 2086s :/ [01:00] the --lsprof output would be interesting [01:00] THen fetched for 9000s. [01:01] wgrant: that is a lot of time spent to figure out it had to fetch everything... [01:02] wgrant: did you fetch into the repo? [01:02] if you fetch into a new branch with no repo you can skip that part i expect [01:02] mwhudson: it went straight to insert stream here [01:04] mwhudson: I did, yeah. [01:04] So I pay 2000 seconds because it retrieves the entire history map to check for anything I already have? [01:04] something like that [01:04] Yay [01:07] Still, at least in-branch operations are pretty fast. [01:07] the smart server protocol should and could be a lot faster [01:07] s/faster/smarter/ [01:08] As mwhudson showed yesterday, a completely dumb fetch took it down from 70 minutes to 37 seconds. [01:13] mwhudson: hi [01:14] bzr info bzr+ssh://bazaar.launchpad.net/~gz/bzr/trivial_testskipped -Dhpps seems to hang on the server side [01:14] wgrant: I guess it's partially a bandwidth usage/cpu usage/latency tradeoff [01:16] mwhudson: i don't suppose we can get a traceback of the production server process? [01:17] poolie: i expect so [01:17] spm: can you copy ~codehost/.bzr.log somewhere poolie and i can see it? [01:17] poolie: hm, it works for me [01:18] i guess i'm seeing the hosted area of the branch though [01:18] mwhudson: sure, one sec [01:20] mwhudson: poolie: chinstrip:~spm/bzr.log.save.gz [01:22] can we easily tar up a copy of the hosted disk directory for that branch? [01:22] and attach it to https://bugs.edge.launchpad.net/launchpad-code/+bug/562672 [01:22] Bug #562672: info on broken? branch hangs on server side [01:26] poolie: spm can, i think [01:29] mwhudson: /srv/bazaar.launchpad.net/push-branches/00/04/f5/1b/ ? [01:33] spm: yeah, and the mirror copy i guess [01:34] yes [01:34] oki [01:35] /srv/bazaar.launchpad.net/mirrors/00/04/f5/1b: No such file or directory [01:35] i think he deleted it [01:35] oh well [01:37] ew. that's a 43Mb tgz. do you really want that attached to the bug? [01:37] I ask as I'm not sure I'll be able to attach it without getting a timeout... [01:37] sure, why not? [01:37] erk [01:37] well, maybe put it on people.c.c. and attach the url? [01:37] been down this path before [01:37] sure [01:39] mwhudson: My local branch has been going for 33 minutes, and has so far copied 99MB. [01:39] wgrant: speed demon [01:43] * mwhudson lunches [01:44] poolie: 15 min eta on that tarball landing on people.c.c.... 27% thru... [01:47] spm, from inside the dc? [01:47] or did you copy it down to canberra? [01:49] poolie: messing around within the dc tends to be sufficently confusing; it's easier to pull locally. if it was 100Mb? yeah, I'd setlle for the hassle and do it right. [02:03] So here's a conundrum. Why would ~vcs-imports be granted launchpad.Admin on IProductSeries? [02:06] maxb: to be able to set the linked branch? [02:06] seems a bit overkill though [02:08] oh. that makes sense... in a sort of sledgehammerish way [03:50] sigh [03:51] when your changes don't seem to be taking effect: check that you're running the tests in the right branch [03:51] :-) [04:38] wgrant: did your clone of the kernel import complete in the end? [05:42] spm: how feasible is it for an appropriately authorized person to look at launchpad's http access logs [05:42] for the apps, not for bzr or other things [05:43] poolie: easy I guess? in a technical sense. The hard part'd be the "authorised person" [05:43] who could do that? or who could ask for it to be done [05:43] I have no idea.... I assume James is involved somewhere; but Francis'd be a good start. [05:43] this is re bug 537432 [05:43] Bug #537432: Save Playlist button does not point to (existing) playlist file

[05:44] mwhudson: It did. [05:44] wgrant: did you see how long it took? [05:45] poolie: ?? I'm not sure I follow the connection there? [05:45] mwhudson: I ran it with --lsprof, so it doesn't really count. [05:45] I am running it again now without it. [05:45] spm, that's cause there isn't one :) [05:46] bug 557432 is better [05:46] Bug #557432: Hot Bugs algorithm works poorly for some projects, making it a bad default listing [05:46] just wanting to know what sort ordering people use most often [05:46] mwhudson: But it took about 96 minutes. [05:47] poolie: I'm guessing newest first. [05:47] poolie: ahh. that'd help then. :-) [05:47] wgrant: i'm guessing most-recently-touched [05:47] but yeah [05:47] Hm, true, or that. [05:47] But probably not heat. [05:48] poolie: ahh. i see. in that case, probably easiest if one of us runs the script directly; the logs are... um. not small. so moving the script to the logs would be a better idea. [05:50] poolie: vaguely related - we did similar with loggerhead/codebrowse a while back. The additional point is that we ran a sanitizer from CERT over the logs; as well as some home grown filtering to strip other bits that were sensitive. So for future ref I guess. we could do similar. [05:50] i think in this case we'd just want the aggregate counts, so there should be nothing sensitive [05:50] I agree [05:51] to write the script one would need to know the precise formats and ideally have a little sample [05:51] wgrant: oh [05:51] though it may just be whatever's in devel's apache config [05:51] wgrant: i hope you kept the output -- can you attach it to the bug? [05:51] mwhudson: I have the lsprof output, yes. [05:51] * wgrant attaches. [05:52] given you're dealing with URL's and stock apache - the URL's can be obtained by pretty much anyone who understands what that bug report is talking about. (I don't - exactly jic you were wondering - get the gist, not the fine detail :-) ) [05:54] spm, my question was, what field of the log is the url [05:54] but i guess i can just assume 'it's the one that looks like a url' unless you're also logging referers or something :) [05:55] heh; is before the referrer. ip - - date tz "type url (http ver)?" ... [05:55] from memory.... [05:56] generally is the 1st '"' field, sometimes the 2nd... /me goes to check. [05:58] yup that looks right. I know some of our logs use an earlier '"' field. one to be wary of. [05:59] type being the http method, like GET? [05:59] and the url is absolute? [05:59] 1.2.3.4 - - [13/Apr/2010:06:39:50 +0100] "GET /@@/footer-background.png HTTP/1.1" 200 404 ..... <== by way of example [06:00] or: 1.2.3.4 bugs.launchpad.net - [13/Apr/2010:06:39:50 +0100] "GET /@@/edit HTTP/1.1" 200 5100 .... <== for otehr domains not specifically logged on their own [06:00] url is always absolute, yes. [06:01] ah but it's only the path part, not including the hostname [06:01] ok [06:01] right [06:02] * mwhudson spots awk lumbering over the horizon [06:02] i'd suggest 1 other part to look for - get a breakdown by Agent. it's not accurate, but *can* be instructive. if only to help identify noise to strip away. eg "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" [06:03] mwhudson: heh; I'll just shove my logfiltering scripts at you guys and let you search'n'replace :-) [06:04] mwhudson: poolie: ancient robot filter: https://pastebin.canonical.com/30558/ [06:04] as in ancient script; not ancient robots. [06:05] Can someone please land lp:~wgrant/launchpad/bug-344165-i-hate-buildd-sequencer? PQM rejected it while in testfix last night. [06:06] wgrant: can you come up with a more.... descriptive branch name? :-P [06:06] spm: http://www.deviceguru.com/files/heath_he-robot.jpg [06:07] >> if ($6 !~ /bot([-./)(_s@ ]|$)| [06:07] heh [06:07] wow, that bug report is written in pure cprov :) [06:08] poolie: I wrote that and still can't figure out what the regex means :-D [06:08] wgrant: so the branch passed ec2? [06:09] mwhudson: Yes. [06:09] it's a steamroller threatening a wombat? [06:10] wgrant: sent to pqm [06:11] okis. heading for some lunch and to let corbs run around and burn off school holidays energy. bbl. [06:11] mwhudson: Thanks. [06:11] spm: You're still in school holidays up there? [06:11] just started end of last week [06:11] Oh. Ours ended then. [06:12] yes. well. we can't have all states on the eastern seaboard as a 1st step align on school holidays! that'd be too logical! <== come and see the cynicism on display! step right up! [06:13] mm [06:13] anyways, me and my cynicism are bbl :-) [06:13] * mwhudson tries to remember how to turn statement logging back on in postgres... [06:13] mwhudson: /etc/postgresql/8.3/postgresql.conf [06:14] log_statement='all' [06:14] wgrant: ta [06:18] hmmm [06:18] INSERT INTO Branch is blocking here [06:18] Awesome. [06:21] stub: do you have any tips for seeing why a particular sql statement is hanging? [06:22] Look at pg_stat_activity to see what else is runnin [06:22] g [06:22] i think i found it [06:23] stub: all the other connections were [06:23] but i guess one of them had taken a lock [06:24] If they were , they didn't have a lock. [06:24] ' in transaction' might [06:24] ah yeah, one was in transaction [07:09] mwhudson: is there an LP API call where i can pass in a url and it gives me the branch-if-any ? [07:10] mwhudson: I might have an http url, or a bzr+ssh one [07:20] thumper: ^ wgrant: ^ [07:20] it might be a mirrored branch too === dpm-afk is now known as dpm [08:14] good morning [08:36] lifeless: there's something like that at the python api level i think, getByURL ? [08:37] don't know if it's exported [08:37] mwhudson: could you peek for me? [08:37] lifeless: https://edge.launchpad.net/+apidoc/devel.html <- it is exported [08:41] mwhudson: thanks! [08:56] whats the 'right' service root to use at the moment ? [09:00] lifeless: LPNET_SERVICE_ROOT, version='1.0' for stability or EDGE_SERVICE_ROOT, version='devel' for tracking changes I would say [09:32] james_w: so you need to pass two parameters these days ? [09:32] james_w: the code examples on the wiki are stale then [09:35] lifeless: you don't have to, version has a default [09:39] ok [09:40] what is it ? [09:52] lifeless: depends on the version of launchpadlib [09:52] 1.0 currently I believe [10:17] mthaddon: ping [10:19] mthaddon: spm: http://paste.ubuntu.com/414192/ - I'd like to know if thats clear enough docs on setting up LP API keys for pqm [10:30] Interesting. [10:31] With Python 2.5, TALES obeys SourcePackageName's __unicode__. [10:31] With Python 2.6, we instead get '' [10:40] is that with an exception or regular object? [10:40] there were some changes in that area [10:40] lifeless: Regular object. [10:40] bigjools: Isn't the explanation in bug #562451 incorrect? [10:40] Bug #562451: developer-membership-board should be able to edit ArchivePermissionSet [10:41] AFAICT, the TB can edit it because they are in ~ubuntu-drivers, which owns the primary archive, so they can do just about anything (which is pretty broken). [10:41] The only methods restricted to launchpad.Edit on ArchivePermissionSet are packageset-related, which is also pretty broken, but not the main problem here. [10:43] wgrant: yeah I was going to comment on that - the code is still wrong, but for other reasons [10:43] I mean "wrong" for philosophical reasons [10:44] AFAICT, it should all work if newPackagesetUploader and deletePackagesetUpload are moved to the block, and the DMB added to ~ubuntu-drivers... but I really don't think most of the members of ~ubuntu-drivers should have the privs they do now... [10:46] yes - see the team description [10:46] In fact, their current permissions seem to somewhere between rather stupid and horrifyling dangerous. [10:46] I am leaning towards the latter. [10:46] Yes... but that has been there for a number of years. [10:54] lifeless: No other ideas? [10:55] Oh gah, I'm sure I remember investigating __unicode__ vs. __str__ issues last python migration push, but I've forgotten the details :-( [10:56] The easy fix is to alter the expression to take /name, like all but two places in the code already do. [10:56] I am tempting to take that path. [10:56] Er, tempted. [11:00] Morning, all. [11:07] wgrant: ? [11:07] wgrant: oh right, -_unicode_- [11:08] I think using explicit accessors is clearer [11:08] I could look at CPython if you're stuck [11:08] Certainly. I'm just a bit suspicious that something deeper has broken that I might be about to hide. [11:08] * wgrant will trust the tests and just add /name. [11:23] I wonder if launchpad-code will blowup if i attach 40 MB comments to it ? [11:29] lifeless: don't know about launchpad-code, but in malone we block comments above reasonable size [11:36] lifeless: 50000 bytes is the limit I think, or 50000 characters. [11:37] ... in malone. [11:41] allenap: intellectronica: So, I guess we need an attachment facility for merge proposals. [11:41] I'll file a bug requesting that. [11:42] lifeless: sounds like a useful thing to have. as a short-term solution what we usually do it attach files to the bug corresponding to the MP and link to the attachment from a comment. [11:43] intellectronica: if there is a bug ... [11:44] lifeless: There already is one. [11:44] lifeless: You just have to submit them by email. [11:44] wgrant: for attachments ? [11:44] Yes. [11:44] lifeless, intellectronica: Actually, I think the limit is on Bug.description (sane_description CHECK constraint). As far as the data model is concerned, it might be okay for comments to be bigger. [11:44] If you attach a diff, it will even render it nicely inline. [11:44] wgrant: oh, I've just sent a mail asking for them again then. [11:45] wgrant: whats the # ? [11:45] allenap: oh, didn't realise that. that's a mis-feature, i think. [11:45] wgrant: and what do you mean 'you just have to submit them by email' ? [11:45] lifeless: As in it's already implemented. [11:45] lifeless: There is no UI to add attachments, but if one is attached to an email then it will work. [11:45] intellectronica: I think there /might/ be a comment limit, but only implemented in the app. [11:46] wgrant: I can't see them on https://edge.launchpad.net/+apidoc/devel.html#branch_merge_proposal [11:46] lifeless: The exposed API is not even close to complete. [11:46] wgrant: care to earn brownie points with me ? :> [11:47] * wgrant has a few too many things on his plate already. [11:48] wgrant: if you can bear the agony, I'm happy to learn stuff [11:48] lifeless: Why do you want it exposed on the API? [11:48] wgrant: so that PQM can attach failure subunit streams, which can be many MB in size [11:49] Ahh. [11:50] Hm, actually, only diff comments are rendered at all at the moment. Others are stored, but rendered only as HTML comments. [11:51] And... hmmmmmmmmmmm. [11:54] Is anyone knowledgeable in bizarre vcs-imports behaviour around? After being reviewed, the initial import of https://code.edge.launchpad.net/~maxb/irssi/trunk was executed 5 times immediately consecutively before it eventually "stuck" and completed for-real [11:55] maxb: bzr-svn and bzr-git initial imports are now done incrementally. [11:55] 1000 revisions in each attempt. [11:55] oh... [11:55] the UI is somewhat mystic about this [11:55] Hence the grey tick the first few times, which should have an informative tooltip. [11:56] (insufficiently obvious, but at least present) [11:56] true (on both counts) [11:59] mthaddon: did you see the pastebin mentioned above ? [11:59] wgrant: and your conclusion on attachments? [12:00] * mthaddon looks [12:00] lifeless: that seems fine [12:01] wgrant: if you're interested in what I'm doing, have a look at lp:pqm - the recent commits [12:01] mthaddon: cool [12:01] mthaddon: I intend to ask spm to try a new pqm for bzr tomorrow (and roll it back if it won't work with the current launchpadlib) [12:01] mthaddon: I wanted to check the docs would be clear enough [12:23] mwhudson/thumper: Either of you still around? [12:30] lifeless: it looks to me as though exposing attachments on merge proposals would be at least one day of solid effort on it [12:31] james_w: thanks for estimating it, thats good to know [12:31] it's possible it would be quicker, but it's all a little odd, and so isn't just a matter of writing tests and slapping exported() on it [12:33] james_w: What do you see as being odd about it? [12:34] you call a method getAtttachments that iterates through chunks on messages and checks their mime type and then returns two lists [12:34] You'd just need a bug-esque addAttachment (and probably linkAttachment). [12:34] Ah, right, for reading them. [12:34] Bugs do the same, but return only one list. [12:34] so to add an attachment you need to add a message with a single chunk [12:35] Right, which is what Bug.addAttachment does. [12:35] so an API approach needs only addAttachment; the web display will need that logic [12:35] Isn't it? [12:35] Oh, it's not. [12:35] Odd. [12:36] That actually uses a separate BugAttachment table. Strange strange strnge. [12:37] oh, it does attach it to a message though, so not as different as I thought [12:37] once again, yay for reinventing comments/attachments/etc. [12:37] There's already a factory method to create the necessary message (I used it a few minutes ago), so it's probably just a matter of shuffling it around. [12:38] yeah [12:38] And maybe strangling the next person who invents a new comment system. [12:38] oh, the description of the attachment is actually the comment that it is attached to? [12:38] interesting [12:38] Hm? [12:39] In which system? [12:39] bugs [12:39] IBug.addAttachement takes comment=Text() [12:40] That's the text of the comment to be attached to, right. [12:40] ah, I'm talking nonsense, ignore me [12:41] I was confusing comment and description [12:41] s/description/title? [12:41] It's title in the DB, at least. [12:41] description in the API [12:41] * wgrant is confused now too. [12:41] Yay. [12:42] but yes, code attachments are sufficiently different that there's not a great deal you could take to implement code attachment addition [12:42] I forgot the two systems were so different :/ === mrevell is now known as mrevell-lunch === matsubara-afk is now known as matsubara [13:41] gmb, is bug 546774 qa-able really? Or just passing a test run is sufficient for this? [13:41] Bug #546774: BugWatchActivity.result should be NOT NULL

[13:42] deryck, Well, we can quickly QA the db patch. I'll ask a LOSA once one comes free. [13:42] deryck, But actually, there's probably no point, come to think about it [13:43] deryck, I think we can mark it qa'd [13:43] * gmb does [13:43] gmb, yeah, that's what I wondered, if it really had any value. Thanks! [13:45] heh, we've emptied the landed/qa queue and filled it up again. [13:45] at least we're being productive [13:45] danilos: Can you provide a log of your 'make run' 'pydoc2.4' issue? FYI, the problem is that somehow the value of PYTHON_VERSION isn't being passed down from the root Makefile to the pygettextpo Makefile [13:47] maxb, sure, though after a successful make it doesn't complain anymore so I am doing it in a new branch [13:47] maxb, btw, it dies on the 'make' rather than 'make run' [13:47] maxb, and the output I have seen was totally uninteresting (though, might give you a tip: it happens in pygettextpo) [13:48] maxb, it seems it might have been stale sourcecode as well [13:50] maxb, yeah, it works just fine now [13:50] The pygettextpo sourcecode branch contains PYTHON_VERSION = 2.4 in its Makefile. This isn't a problem, because when the root makefile recurses into sourcecode, it passes its own PYTHON_VERSION as an override to the sub-make [13:51] maxb, well, that's where it was trying to execute pydoc2.4; it doesn't anymore, perhaps I need to change some timestamps so make wants to run it again === mrevell-lunch is now known as mrevell [13:53] wfm in a completely clean environment, so I think it was just staleness hitting you [13:59] maxb, yeah, I've tried after doing 'make clean' in pygettextpo and it still works and even rebuilds docs correctly [14:22] jelmer (author), rockstar (reviewer): Regarding the cscvs "Set the 'converted-from' revision property." change landed on 2009-12-07. It appears it never got used, because sourcedeps.conf was not updated. I now have a later change to cscvs I want activated. Can you advise on the suitability of that change to go live? [14:23] maxb: heh, whoops [14:23] maxb: I guess any external users of cscvs would've been using it though [14:23] Do any of those exist? :-P [14:23] maxb: There's no reason for the converted-from change *not* to go live [14:24] I'm a bit unclear on the policy for bumping revnos in sourcedeps.conf. Does it need a review, or do I just need to get someone to rubberstamp and land? [14:28] noodles775, ping [14:29] deryck: hi [14:33] maxb: I *think* you just need a review given the original change has been reviewed by a launchpad reviewer already [14:34] maxb: it's probably different for projects in sourcecode that aren't maintained by launchpad-pqm [14:35] ok, I'll put a branch up that bumps the revno to include yours and my changes, and leave it to OCR discretion whether to r or rs [14:45] * wgrant points at bug #562943, and bugs Bugs people to unbreak notifications [14:51] wgrant, that bug is on MPs, not bugs, or I'm reading it wrong? [14:51] deryck: The bug is Code, but it is probably Bugs' problem that there will have been no notifications about it. [14:51] Security bugs against Launchpad go into a black hole. [14:52] Nobody except the reporter and a list that is never read get notified. [14:52] ah, I get what you mean now. Yeah, that is likely on us. [14:52] wgrant, is there a bug about that then, about the black hole? [14:53] deryck: It's probably not so much a bug as a config issue. [14:53] A config issue which exists because the Launchpad notification system is pretty awful. [14:53] wgrant, right, agreed. But still would be nice to have a bug to track the issue and see if better general solution couldn't be found. [14:56] intellectronica, can you do a quick pre-imp about bug 556499 now? [14:56] Bug #556499: Set user expectations for Ubuntu bugs after user reports a bug [14:58] deryck: sure. just give me a sec to get on skype again [14:58] intellectronica, great, same for me. thanks! [15:00] abentley: Bug #562943 may interest you. [15:00] deryck: ready when you are === Ursinha-afk is now known as Ursinha [15:08] hi gary_poster, I know you're busy now, but for you to answer later :) I've seen about 200 daily oopses like this one: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1564CEMAIL1, I wonder if it's the same problem as bug 54005 but with a slightly different symptom? [15:08] Bug #54005: bugs.launchpad.ubuntu.com email ending up in Launchpad mailbox [15:13] Ursinha: I have no idea, unfortunately. stub, are you around and available to look at what Ursinha wrote and share a guess? [15:13] * stub has a look [15:14] I'd say the OOPS system is rooted [15:18] stub, uh? [15:18] The OOPS is almost entirely blank. There is an email listed though, which if I examine looks like it was sent to noreply@ubuntu.com but delivered by the mail system to launchpad@mail.canonical.com [15:18] Have a look at that oops [15:18] Traceback: None None Traceback (most recent call last): None [15:19] I guess it is a handled error that we are just dumping to an OOPS for reporting? [15:19] stub, I did see oops. I didn't know what to think of it either. So you think we have a security compromise on the OOPS machine? matsubara, any thoughts? [15:20] I don't see a security compromise anywhere. [15:20] sorry, then what did you mean by "rooted" [15:21] * gary_poster looks up australian definitions :-) [15:21] * gary_poster thinks he understands [15:21] "broken" [15:21] yes :) [15:21] :-) k [15:21] hehe [15:22] I think it's the same issue described in bug 54005 [15:22] It looks like a wiki update email was sent to launchpad@mail.canonical.com, and our mail processor doesn't know how to handle it. [15:22] Bug #54005: bugs.launchpad.ubuntu.com email ending up in Launchpad mailbox [15:22]