[00:00] the text formatter is broken then, it linkifies bug\nNNN [00:06] bigjools: It deliberately does that, to cope with wordwrapped emails [00:06] see my text in bug 1081834 [00:06] <_mup_> Bug #1081834: When setting a milestone on a bug task, the edit icon ends up as the "broken graphic" < https://launchpad.net/bugs/1081834 > [00:07] I say tough titty if it's word wrapped [00:07] coz now you have a broken link [00:08] It's probably less bad to have that extra link than it is to miss some [00:09] I disagree [00:09] Why? [00:09] You can just not click on that link :) [00:09] tell that to oops reports [00:09] you can just type a bug number too [00:10] But I can just as easily generate a bad link malevolently by typing "bug 2" in a comment [00:10] Those sorts of things should be (but aren't presently) excluded from OOPS reports [00:10] why is it even linked if it doesn't exist? [00:10] Because user error can cause them [00:10] it's ab ug [00:10] should not be linked [00:11] It's turned grey by JS if it doesn't exist. [00:11] a start, I suppose [00:11] Users can type bad URLs [00:11] They can type bad bug numbers too [00:12] well it doesn't go anywhere, I thought it was a real link, so it's opk [00:12] It is a real link until the JS detects that it doesn't actually exist [00:12] Which isn't ideal, but I think it's a reasonable compromise [00:12] so robots will still click it then [00:12] Yes [00:13] I am sure there was code in the formatter to not linkify non-existent bugs [00:13] There was, but we removed it [00:13] Because it was completely terrible [00:13] let me guess ... [00:13] Making a separate query for each bug reference [00:13] yeah [00:13] And it's difficult to do otherwise without a multi-pass template system, which TAL is not [00:13] I remember it being a problem on changelogs [00:14] Anyway [00:14] "bug 2" is just a special type of URL [00:14] Users can type bad URLs [00:14] and when I wanted to remove it to fix performance, all of Ubuntu screamed like a thousand voices in the night [00:14] We still linkify them [00:14] Why not do the same with bugs? [00:14] It's not harming anyone [00:15] ok, you're on maintenance :) [00:15] 404s are a discretionary sort of OOPS, since users will always be able to generate them simply by typing a bad URL somewhere on LP [00:15] * StevenK stabs gina.txt and twists the knife. [00:15] They don't unconditionally fall under ZOP [00:15] ok [00:15] And they don't degrade user experience [00:15] => meh [00:16] well at least you thought about it [00:17] The linkification of bullets isn't ideal, but there's not much we can do about it without breaking other cases [00:17] Of course, if you used punctuation properly it wouldn't be a problem :P [00:18] => meh [00:18] DWIM etc etc [00:18] Heh [00:24] wgrant: Hmmm, what about bug 837563? [00:24] <_mup_> Bug #837563: language-pack-exporter script raised a KeyError < https://launchpad.net/bugs/837563 > [00:25] I thought OOPS prefixes were long dead [00:26] StevenK: Sure, the REPORTIFSEEN part of that bug is no longer relevant [00:26] The underlying OOPS is [01:47] hmmm, why would ^\s not work in my sed expression [01:48] \s as in 'any whitespace' ? [01:48] yup [01:48] ^\s should be leading whitespace [01:49] yet it matcheth not [01:49] \\s ? [01:50] nope [01:50] removing the ^ matches but I want to be more specific [01:51] [[:space:]] [01:52] \s is a perl-regex-ism [01:52] it's in the sed man page :) [01:52] not in the one I have here it's not [01:52] anyway, [[:space:]] is the POSIX form [01:53] ^[[:space:]] didn't work [01:53] full example please [01:54] sed -ri "s|^[[:space:]]generator:\shttp://.*:5243/| http://bar:5243/|" test.tmp [01:54] you still have a \s there [01:55] and I think perhaps you mean [[:space:]]* for zero or more whitespace characters? [01:55] makes no difference if I change it [01:55] [[:space:]] (or \s in regex engines that support that) means exactly one whitespace character [01:55] I want a + actually, I'll change it [01:55] \o/ [01:55] thanks cjwatson [01:56] yw [01:56] I really hate regex [01:56] bigjools: You now have two problems. [01:56] exactly [01:56] Or three, if you're also dealing with mod_rewrite [01:57] heh [01:58] wgrant: Can't find OOPS-8e259d8c3b45707c6a37f986d3c8127d on carob's filesystem. :-( [02:00] wgrant: I wonder if we can just destroy the branch puller [02:04] mwhudson had plans :) [02:05] and doesn't now ? [02:09] i just don't have time now [02:09] wgrant: actually i futzed a little on the flight to linaro connect about this and wanted to talk to someone (probably you!) about this [02:11] wgrant: my plan went something like this: [02:11] set up a launchpad user on each importd [02:11] have a fkey from codeimportmachine to this user [02:11] change _canWrite to something like this: [02:11] + elif branch.branch_type == BranchType.IMPORTED: [02:11] + job = branch.code_import.import_job [02:11] + if job.machine: [02:11] + return job.machine.user == requester [02:11] + else: [02:11] + return False [02:11] mwhudson: so did you decide to stay w/Linaro or move back into Canonical ? [02:13] mwhudson: That's probably the cheapest way to do it, but ideally we'd also solve stacking at the same time [02:13] change things around so that a lp:// url rather than a escudero url gets passed to the worker [02:13] wgrant: "solve stacking"? [02:13] mwhudson: Stacking imports would sometimes be nice. [02:13] oh right [02:14] Which means access to more than just that branch [02:14] Which gets messy [02:14] well [02:14] imports would end up stacked on the project trunk in this case [02:14] but they would get imported from scratch first [02:14] which would not be so ideal [02:14] Well [02:15] They might end up using /+branch-id, which wouldn't stack [02:15] oh true [02:15] and indeed, that would be easiest [02:15] And safest [02:16] so the fix is to make pushing to +branch-id stack? [02:16] Maybe [02:16] Or just ignore stacking [02:16] i don't think attempting to solve stacking at the same time seems especially sensible [02:17] Indeed, perhaps not [02:17] But it should be kept in mind as a future consideration, perhaps [02:17] once imports don't need the puller, converting mirrored to imported branches should be trivial [02:18] and then one can rm -rf lib/lp/codehosting/puller [02:18] which would get you lots of LOC credit :-) [02:18] Yep [02:18] That's why I haven't converted mirrors yet [02:18] Because of the escudero mess [02:18] right [02:19] someone should delete all the codeimportevent mess too :( [02:19] that was mostly ddaa really but i feel bad for letting it happen [02:19] What's the mess there? [02:19] I admit I don't know those corners of the model well [02:19] well, the whole concept is stupid [02:19] not stupid [02:20] but should have been YAGNIed before implementation [02:20] Oh [02:20] That's distinct from codeimportresult... I... seee [02:20] yeah [02:20] thats hardly unique in LP [02:21] Ah [02:21] So CodeImportEvent is an audit log, basically [02:21] yes [02:21] that you mostly can't read [02:21] auditor being the now extant answer for that [02:21] Well [02:22] Except it isn't extant yet :) [02:22] ENODEPLOY ? [02:22] and that means that standard methods are not used to mutate them, which means that e.g. field validators don't run [02:22] Yeah [02:22] wgrant: https://code.launchpad.net/~mwhudson/launchpad has a few branches at the top if that's useful at all [02:23] i can't really remember how far i got, i was looking at changing the worker invocation code when i got discouraged i think [02:24] Hmm, have you made a terrible mistake... [02:24] I guess we'll see [02:25] (pushing a series of branches which will each want to create the same new revisions, and they'll be scanned concurrently now...) [02:25] oh [02:25] heh [02:25] they certainly don't seem to be being scanned [02:25] It takes a 80s at best to do a fresh scan of an LP branch nowadays [02:26] Often a few minutes [02:28] Yeah, they all OOPSed [02:29] sigh [02:30] The branch scanner does not win any points for considering locks or using sensible transaction sizes :( [02:34] lifeless: We don't have the staging instance deployed yet, let alone think about production. [02:46] wgrant: http://pastebin.ubuntu.com/1376306/ but I'm not sure how to test it [02:47] Um no [02:47] Transaction management does not go in DB model classes :) [02:48] Wrap the generateIncrementalDiff in BMPJ run() ? [02:50] wgrant: http://pastebin.ubuntu.com/1376309/ [02:51] That I can test, if I can patch out generateIncrementalDiff() [02:51] Well [02:51] We in general need a strategy for handling conflicts in jobs [02:52] eg. what mwhudson just ran into [02:52] Ignoring all IntegrityErrors is probably not it :) [03:18] wgrant: Hmmm, it looks like cronscripts/rosetta-export-queue.py already uses the slave DB and based on the logs it isn't getting reaped. bug 504821 [03:18] <_mup_> Bug #504821: poimport (export)_uses a single long transaction, gets reaped < https://launchpad.net/bugs/504821 > [03:20] It's still vile [03:20] But indeed, I haven't seen reaping in quite some time [03:21] wgrant: Downgrade or close? [03:21] Invalid until we have reason to believe otherwise [03:24] It's still something we need to adjust eventually, but it's not causing any trouble today [03:24] And it's not actively a problem until we ratchet our paranoia up another notch [03:26] StevenK: In what circumstances does one not have an id? [03:31] I'm guessing when we get a error page or something like that [03:31] There are a whole stack of oopses on neem [03:31] Right, they certainly happen [03:31] But we should work out why [03:32] Blindly bypassing an error condition without understanding it is not the way to fix criticals, sadly [03:32] (unless we have no other choice) [03:34] I just noticed the OOPSes while looking if there was a langpack export OOPS [03:35] Checking if they're all the same bugtracker [03:35] Sure [03:35] It's been happening for a while [03:35] And it's likely to be a simple fix [03:36] But we need to understand why [03:36] using the slave db is now pointless [03:36] as transaction bloat is propogated back to the master [03:37] lifeless: Pointless for bloat [03:37] Not pointless for load [03:37] not terribly interesting either ;> [03:37] (not that load is an issue on any system today) [03:37] right [04:00] Oh, hah [04:00] mercurial has replaced their roundup instance with bugzilla and are redirecting [04:00] Ah [04:05] wgrant: Right, there's mercurial fixed, two link farm bugtrackers deleted and I yet to figure out what setuptools is doing [04:06] :) [08:30] good morning [09:15] aloha [10:22] cjwatson: ello how do you want this triaged, https://bugs.launchpad.net/launchpad/+bug/1081860 [10:22] <_mup_> Bug #1081860: uploads to oneiric/partner are redirected to oneiric-proposed/partner < https://launchpad.net/bugs/1081860 > [10:29] czajkowski: I stole it for ubuntu-archive-tools [10:29] So you won't have to care :) [10:29] oki dokie :) [10:29] thanks colin === matsubara is now known as matsubara-lunch === almaisan-away is now known as al-maisan === al-maisan is now known as almaisan-away === matsubara-lunch is now known as matsubara [17:31] cjwatson: me again :) can you moderate a mail if you get a chance later on to devel [17:32] really find it hard to type devel without going devil :/ [17:33] czajkowski: done [17:34] cjwatson: thank you === cyclicflux is now known as Guest70182 [18:18] is there a secret way of only retrieving certain keys when GETting Launchpad API resources? [18:18] some code_review_comments are really, really big. [18:20] further, the size of the download is 2x the size of the comment, because as_quoted_email is included in the dict [18:21] also, they both get stored in memory as unicode objects, which take up 4x as many bytes as the string has characters [18:21] bringing us up to 8x [18:22] then, if you are getting it as part of a collection, the naive way of using it (looping through), will give you a copy of the data in the collection _and_ a copy in the entry object [18:22] taking us up to 16x [18:23] for this use case, I don't care about the comment body at all. All I want is the dates. [18:35] jml: most of the devs are off today as USA based [18:42] czajkowski: so I should try to avoid having hard problems today and tomorrow? [19:30] jml: most are back tomorrow tbh, or leave a message for wgrant he'll be back online shortlyish [19:31] jml: adeuring will be back jtv bigjools, or gmb [22:26] jml: Well, you can retrieve individual attributes directly, but you can't retrieve a subset of elements that isn't either the whole lot or a single one. [22:26] lazr.restful unfortunately provides no more advanced facility [22:26] jml: You should upgrade to Python 3.3 :) [22:28] jml: Are these really, really big code_review_comments full of test results? [23:15] abentley, yes