[00:15] Project devel build #983: STILL FAILING in 5 hr 47 min: https://lpci.wedontsleep.org/job/devel/983/ [00:17] !!! [00:17] Jenkins is back! === nhandler_ is now known as nhandler [00:37] Project db-devel build #816: STILL FAILING in 5 hr 44 min: https://lpci.wedontsleep.org/job/db-devel/816/ [00:53] Hello everybody. I believe trunk is still broken. This fixes the last test (monkey-see monkey-do from what wgrant submitted for fixing the other tests) but I don't know if it is correct. http://pastebin.ubuntu.com/672811/ Can anyone verify? If so, I'll do another testfix. [00:54] lifeless, wallyworld, are either of you around and able to verify correctness? [00:54] FWIW, this runs the failing test: [00:54] ./bin/test -cvvt lp.bugs.model.tests.test_bugtask.TestValidateTransitionToTarget.test_sourcepackage_to_sourcepackage_in_same_series_works [00:55] gary_poster: i'll have a look [00:55] thank you wallyworld [00:55] np [00:55] gary_poster: i think wgrant submitted a test fix - are you saying it's still broken? [00:55] or is this a different issue? [00:56] yes wallyworld. He missed one test. I copied his fix pattern to the remaining one. [00:56] ah, right [00:56] perhaps we should nuke bb [00:56] No LOSAs [00:56] still? [00:56] Not in this timezone [00:57] s'alright. If I land a testfix it will immediately run the next test, with only an email compaint. [00:57] I mean, immediately after the current one fails [00:57] so there shouldn't actually be any outage for you guys [00:57] ok, thanks! [00:58] the pastebin looks ok to my untrained eye. running the test to verify [00:59] wallyworld, I'll treat this effectively as an r= and then self-approve :-) [00:59] (once you run the test [00:59] ) [00:59] ok, doing it now [01:00] wallyworld, I still intend to take you up on more of the PyCharm hints (like the bzr plugin). Things have felt a bit up-in-the-air lately so I haven't attempted the full PyCharm switch yet [01:01] StevenK, btw, know that I would have begged for your assistance by name too had I remembered that you were around. ;-) [01:01] gary_poster: whenever on pycharm. bigjools likes it :-) [01:01] gary_poster: the test passes :-) [01:01] cool wallyworld :-) [01:02] thanks for fixing [01:03] test passing: great thanks wallyworld, and I'm taking that also as affirmation that, to your best guess, you think it is not making the test pass...by asserting broken functionality or something. :-) OK, off to submit... [01:03] gary_poster: Haha [01:03] :-) [01:04] And here I was thinking that I was loud and annoying enough to not be forgotten. [01:04] gary_poster: well, don't take me as someone who knows about this bit of the product :-) so my best guess is not necessarily all that good [01:04] but the fix seems like it's in line with the observed failure [01:05] FUUUUUUUUUU More test failures. [01:05] wallyworld, yeah. Fair enough. Here's hoping, then. Right, and it looks like what wgrant did. [01:05] I can't figure out what I broke [01:05] nigelb: what's up? [01:05] wallyworld: two attempts at landing that branch and I have test failures [01:06] nigelb, trunk is broken fwiw, and was broken worse before. If the errors look like "IllegalTarget: Package unique-from-factory-py-line3362-24070 not published in Distribution-24050" then wait a sec and then pull from trunk once I have the last fix in. [01:06] nigelb: some test failures occurred due recently due to some broken code elsewhere. perhpas you are seeing that issue [01:06] do you want to pastebin the failures? [01:06] gary_poster: Indeeed it does! [01:07] nigelb: It happens like that sometimes, sadly. [01:07] nigelb, well, good news, for some definition of good news. ;-) I'll ping you when it's in. Should be <5 min. [01:07] \o/ [01:07] gary_poster: Thanks :) [01:07] nigelb: Did you want your branch tossed at ec2 for the 3rd time? [01:08] StevenK: sec, there's one failure that's fallout from me. [01:08] let me fix that [01:08] doctests *stab* *stab* *stab* [01:08] StevenK: if it's just the distro packaging failures we could lp-land it? [01:10] wgrant, any objections to http://pastebin.ubuntu.com/672811/ ? About to land it? [01:10] gary_poster: Only if you format sp2 correctly [01:11] sp2 = self.factory.makeSourcePackage( [01:11] StevenK, bah, that's PEP8 compliant last time I checked :-P [01:11] distroseries=sp1.distroseries, publish=True) [01:11] If lint complains I'll do it...waiting... [01:11] gary_poster: Huh, that mustn't have come up in the summary... [01:11] Let me check. [01:12] StevenK, linter likes my code more than you do ;-) [01:12] Thanks wgrant [01:12] gary_poster: Bah! :-) [01:12] gary_poster: Argh, no, I just misread. thought it was part of TestValidateTarget. [01:13] wgrant, ok cool, so I'm assuming you approve of what I did. I'm about to lp-land [01:13] gary_poster: That diff looks fine. [01:13] Thanks. [01:13] thank you [01:21] nigelp, merge trunk and all those other ones should go away. good luck. [01:21] nigelb, even :-P [01:23] hehe, okay [01:35] wallyworld: 274 + var filter_msg = Y.substitute( [01:35] 275 + 'Showing {filter} matches for "{search_terms}".', [01:35] 276 + {filter: current_filter_value, [01:35] 277 + search_terms: this._search_input.get('value')}); [01:35] yeees? [01:36] wallyworld: It's not exploitable, as it's only displaying immediate user input, but it is wrong. [01:36] security issue? [01:37] It would be, if search_terms didn't happen to come from a textbox. [01:37] so the problem is? [01:37] Search terms involving HTML will break the world. [01:38] And it's using a forbidden pattern which people will follow. [01:38] well if anyone types in html they deserve it to break [01:38] i just cargo culted the pattern myself [01:38] Right. [01:38] Let's not perpetuate this. [01:38] i dont' see this instance as a huge issue in this case [01:39] but yes, as a drive-by.... [01:40] In the current form it is barely exploitable, true. [01:40] But it is a terrible pattern, and could very easily accidentally become exploitable. [01:40] sure, so next time someone is in the neighbourhood..... [01:41] lifeless: https://code.launchpad.net/~wgrant/launchpad/fix-codebrowse-oopses/+merge/72515 [01:42] wgrant: r=me [01:43] StevenK: could you land https://code.launchpad.net/~nigelbabu/launchpad/4595-upgrade-bug-linking/+merge/71575 again? :) [01:43] StevenK: I want to wait for lifeless, as this was obvious enough from the logs that I think I might be missing some other issue. [01:43] Third time's the charm I hope. [01:55] if anyone types in html> such as web browser developers looking for bugs involving certain kinds of HTML, for example ... [01:55] cjwatson: Surely not! [01:55] Nobody would ever do that. [01:56] Web applications can break in bad ways in some cases. [01:56] All good. === gary_poster changed the topic of #launchpad-dev to: https:dev.launchpad.net/ | On call reviewer: - | Critical bugs: 238 - 0:[#######=]:256 [02:16] wgrant: hi [02:17] lifeless: I tried loggerhead locally and got a pretty obvious traceback. Did that not appear on prod? [02:17] s/prod/qas/ [02:17] qas just ISE'd on us [02:18] I prepped a monkey to get into pdb and see what is causing it - [02:18] ok pages are ok [02:18] but failing pages didn't seem to oops [02:18] its in the -ops history [02:18] http://paste.ubuntu.com/672845/ [02:19] hahhhhhhahhahahaha bloody ha ha [02:19] Yes. [02:19] That stuff didn't make it into qastaging's debug.log? [02:19] no [02:19] Yay [02:19] Shall I land the fix, then? [02:20] hell yes [02:20] if thats s/oopsid/id/ [02:20] It is. [02:20] It even works. [02:20] Sad that this isn't tested. [02:20] thank you & sinzui very much [02:20] But let's unbreak it first. [02:21] lifeless: Do I want a --rollback? [02:21] Since it's already been fixed once, right? [02:21] Let's see the report... [02:23] Ah, there was no --rollback last time. [02:23] I shall put it on this one. [02:33] lifeless: O hai [02:33] lifeless: Can you peer at http://pastebin.ubuntu.com/672806/ when you have a tick? [02:34] wgrant: the cascade thing had me suspicious [02:34] wgrant: I figured I could lp-land a no-op --rollback easier than unlanding one that didn't fix it [02:34] lifeless: I'd like the query to return every BPR that has unexpired BPFs and doesn't already exist in BPRC. [02:36] StevenK: what context is it running in? [02:36] lifeless: Garbo job [02:36] 18 seconds is tolerable to seed the job [02:37] lifeless: wgrant's concern was that the performance is like that and BPRC only has 1 million rows on DF -- it's only going to get worse [02:37] well [02:38] the binarypackagereleasecontents_pkey index will allow fairly efficient determination of bpr's [02:40] the merge join took most of the time [02:40] whats thesecond query ? [02:41] this is inefficient: AND NOT EXISTS [02:41] It's the same query, cold and hot [02:41] (SELECT BinaryPackageReleaseContents.binarypackagepath [02:41] FROM BinaryPackageReleaseContents [02:41] WHERE BinaryPackageReleaseContents.binarypackagerelease = BinaryPackageRelease.id) [02:46] lifeless: Are you fiddling for something more efficient? [02:46] yes [02:52] ah, its wgrant's review day. [02:52] So it is. [02:52] StevenK: did you get a chance to land that? Should I poke wgrant into landing? [02:53] 4595-upgrade-bug-linking => devel [OK] (up for 1:09:13.836563) [02:54] StevenK: Time: 5.961 ms [02:55] http://paste.ubuntu.com/672861/ [02:55] StevenK: this takes advantage of the fact that we expect: [02:55] - most rows to need analysis [02:55] - and we're walking a batch upward [02:56] with analyze http://paste.ubuntu.com/672862/ [02:56] StevenK: aha, thanks :) [02:56] lifeless: Why the first limit 1000? [02:56] * nigelb heads for caffeine and breakfast [02:56] StevenK: thats large enough that we won't get lots of empty batches [02:57] StevenK: its a guess. It just needs to be large enough that we'll get 1 (one is enough) rows most of the time [02:57] lifeless: Won't an empty batch cause it to think it's done? [02:57] yes, the driver logic needs tweaking [02:57] it should say something like 'max id - last seen id < 1000' [02:59] and/or [03:00] always increments the last seen id by 20 if no results were found [03:00] when it exceeds the max id, we're done-done. [03:00] and can switch to a simpler plan using the highest ids and working back or whatever [03:15] lifeless: in case you're interested, I added a first-go patch to https://code.djangoproject.com/ticket/16674 that makes django work with python-oops-wsgi [03:22] wgrant: have you had a chance to look at that buildbot failure? With you, me, and the big J on the blamelist I guess it's not as easy as "oh it's packaging-related, I know who it is." :) [03:22] jtv: It was my fault. It is fixed. [03:22] Oh good, thanks. [04:06] jamesh: I am [04:07] jamesh: do we want a python-oops-django containing the django glue ? [04:07] lifeless: ideally I'd like it if no glue was necessary [04:07] hence the django bug report [04:07] yeah [04:07] me too [04:08] jamesh: thanks for digging into this! [04:08] I'm going to be hacking on more gpg extraction / gpgverifyd today [04:08] but should be back on oopses tomorrow [04:08] and probably on leave monday [04:12] lifeless: On Monday, or from Monday? [04:14] lifeless: you said that some other web frameworks had the same problem of hiding the exception context when producing an error page. Perhaps the best solution would be to patch a few other popular ones and hope that python-oops-wsgi is compelling enough to convince others to patch the remaining ones [04:14] jamesh: that's interesting. [04:14] jamesh: I was just thinking the other day what it would take to use python-oops with django [04:15] (after all the excitement lifeless was making with python-oops :P) [04:18] nigelb: well, hopefully the patch in that bug (or a better version of the same) will make it into a future version of Django [04:19] then the answer is just "wrap Django's WSGIHandler() with python-oops-wsgi and embed in your favourite WSGI container" [04:19] :) [04:20] nigelb: In reference to your G+ post: :shell [04:21] StevenK: I used byobu instead [04:21] with a vertical split [04:21] Cheater [04:21] heh [04:22] Its pretty nifty actually. [04:22] I like commiting often and a terminal right below the editor helps [04:23] Hmm. [04:23] https://bugs.launchpad.net/launchpad/+bug/411409 [04:23] <_mup_> Bug #411409: AJAX "Link to a bug report" could be improved

< https://launchpad.net/bugs/411409 > [04:23] The attachment link there no longer works. [04:24] (I always used to insert odd characters into attachment names to break things... but I think this one used to work) [04:27] Interestingly it works locally. [04:27] The number is the LFA? [04:27] (In the URL, I mean) [04:27] BugAttachment.id [04:28] How much work is that bug? [04:28] Its tempting to pick up [04:28] http://launchpadlibrarian.net/30110664/what%3F.png works [04:29] nigelb: Pickers are not something you want to get into. [04:29] At least not yet. [04:29] Hmm, ok. [04:30] Yes, that picker sucks [04:30] That's not even a picker. [04:30] it should be. [04:30] Do we even have a picker that can search by bug number? [04:30] No. [04:30] We want one that can search by ID/summary/description. [04:31] True [04:31] Can haz FDT now [04:33] lifeless: So I need another number for {B,S}PPH indices -- is it another -1 and goes to db-devel? [04:34] StevenK: That can be done live. [04:34] Once the column is there. [04:34] Wait until then, I suspect. [04:34] Bleh [04:35] We probably need more than just CREATE INDEX binarypackagepublishinghistory__binarypackagename__idx ON BinaryPackagePublishingHistory(binarypackagename) anyway [04:35] Hah, yes. [04:35] (And the correspending SPPH, of course) [04:35] wgrant: Suggestions for what we want to index? [04:35] Depends what you want to make fast. [04:36] The DSP vocab [04:36] The current indexes are pretty arbitrary and useless. [04:36] StevenK: Do you have an example query? [04:37] Given the [04:37] Not handy [04:37] possible new DSP cache table approach, this may all be irrelevant. [04:37] wgrant: Yes, I wanted to talk about that too [04:37] Since we'll have a cache table which has (archive, sourcepackagename, [binaries]) or so. [04:37] We will? [04:37] I hope so. [04:38] Something like DistributionSourcePackageCache, except not stupid and less useless. [04:38] binpkgnames | text | [04:38] Also without that. [04:38] plural -> text [04:38] And not like DistributionSourcePackageInDatabase? :-) [04:38] Uh, yeah. [04:39] StevenK: Have you, sinzui and bigjools discussed this at all? [04:39] I saw bigjools decrying the idea that something could be in a distribution without a publication, but that's about it. [04:39] The thing that sinzui and I have discussed is populating the DSP table for series older than maverick [04:40] And for other series. [04:40] It doesn't need populating for old series. [04:40] It needs reconceiving, reworking, and reimplementing. [04:40] Then you can think about populating it. [04:45] wgrant, are you reviewing today? You might enjoy this one. https://code.launchpad.net/~jtv/launchpad/fix-some-utilities/+merge/72523 [04:46] wgrant: So we have DSP, DSPC, and DSPID. /wrists [04:46] StevenK: Right. We need to talk about how this should work. [04:46] jtv: Sure. [04:46] jtv: The number of selfs in interfaces is disturbing. [04:47] Yes, maybe we should just add a grep to "make lint." :-) [04:47] StevenK: (DSPC is presently updated by a script that takes like 20 hours) [04:48] And DSPID is perpuated by IPublishingSet.newSourcePublication() [04:48] It's not quite clear what should happen to DSPID. [04:48] It possibly still has a place in the world. [04:49] jtv: Approved. [04:49] Thanks. [04:50] StevenK: I think we need to take DSP, DSPC, DSPC and DSPID, all their users, and some other stuff and work out WTF we have and what we want. [04:51] (no the two DSPCs are not a mistake: DistributionSourcePackageCache and DistroSeriesPackageCache) [04:51] I can hear your wrists from here. [04:52] That's funny, I didn't think blood pooling on the carpet was that noisy. [04:54] Is StevenK slashing his wrists again? :P [04:55] * StevenK logs into the ec2 instance that is testing nigelb's branch and injects a few failures. [04:56] gah! [04:56] lifeless: https://bugs.launchpad.net/python-oops-wsgi/+subscriptions <- please unsubscribe launchpad-security [04:56] Anyone can add the subscription, but only a team admin can remove it :) [04:57] hm, isn't there something wrong with that? [04:57] wgrant: So I think DSP is used by a *lot* of stuff -- and DSPID is tied directly into most of it. The two DSPCs, I don't really know [04:59] Merging DSP and DSPID is going to be really fiddly and time-consuming, if it ever happens. [05:00] It may be best to add a flag on DSPID indicating officialness. [05:00] Or just assume that is presence is official. [05:00] But then you'd also need to join against DSPC to do the binary lookup. [05:00] DSPC has the complication that it has an archive. [05:01] So I think you're right. It's a horrid horrid mess, and it needs to die. [05:01] You know what would be cool? [05:01] If Launchpad wasn't a monolithic monster with no separation or interfaces. [05:02] Hah [05:03] Right, DSP fills me with despair [05:03] But if we can work this out, then Soyuz will be much closer to its own little world. [05:03] s/world/insane asylum/ [05:03] Since Bugs will have no fingers left in the Soyuz pie. [05:04] Oh, curses. [05:04] Components. [05:04] And bug nomination permissions. [05:05] * wgrant dies. [05:05] Yes [05:05] I'm making a list, and getting more depressed by the microsecond [05:05] Anyway, let's see what still uses DSPC² [05:05] I think most of its users are long-dead. [05:06] Which DSPC? [05:06] ² [05:06] ie. both. [05:06] They are similar. [05:06] And have the same name. [05:07] OH, WHAT THE [05:07] ? [05:07] The name is slightly different [05:07] DistroSeries vs DistributionSource [05:07] DistributionSourcePackage.getPersonsByEmail(email_addresses) [05:07] SERIOUSLY? [05:07] Pardon? [05:07] lib/lp/registry/model/distributionsourcepackage.py: def getPersonsByEmail(email_addresses): [05:07] It's in there, and the DSP view calls it [05:07] O_O [05:07] You are correct. [05:08] It will die before EOD. [05:08] Can I kill it? [05:08] brb, qannotate [05:08] the_changelog = '\n'.join( [05:08] [spr.changelog_entry for spr in sprs [05:08] if not_empty(spr.changelog_entry)]) [05:09] * StevenK now feels depressed and ill [05:09] That evil is somewhat necessary, unfortunately. [05:12] Icky [05:13] Debian changelogs are, yes. [05:13] If we're logged in, pull every single e-mail address out of a constructed changelog, call that abomdination of a method, and then drop persons that have hidden mail addresses [05:13] *Disgusting* [05:16] It does seem to be the sort of thing that built up slowly over 5 years into an unfathomable monster, which could have been avoided if it had been redesigned half way through. [05:16] Hmm, like something else... [05:16] Launchpad! [05:17] hang on [05:17] why do we drop those people? [05:17] Probably so we can show the email addresses, just unlinked because why. === lifeless changed the topic of #launchpad-dev to: Performance Tuesday | https:dev.launchpad.net/ | On call reviewer: - | Critical bugs: 238 - 0:[#######=]:256 [05:18] well [05:18] couldn't we show the person and not show the email addresses ? [05:18] The changelog is indexed by numerous other places. [05:19] Anybody putting their email address in a changelog but wanting it to remain private is even more misguided than someone who wants their email address to remain private. [05:19] And that is saying something. [05:19] sure [05:20] but thats a separate argument, AIUI we're currently not linking to the persons [05:20] which seems bizarro [05:20] Not necessarily. [05:20] To link them would allow you to determine the owner of an address. [05:20] you can probe for (many) by sending a comment to a bug with a custom from [05:20] s/an address/a private address/ [05:20] Ja. [05:20] over half our users in fact [05:21] Hm? [05:21] does 'private address' mean 'do not show', or 'do not link against from other data sources' [05:21] I think its intended to be the former [05:21] lalalala [05:22] IIRC the UI says 'hide address' not 'pretend it does not exist' [05:22] Hide my email addresses from other Launchpad users [05:22] right [05:25] wgrant: And if we're going to attack DSP/DSPID, what about its bastard half-sibling, DSPR? [05:25] StevenK: It's unrelated. [05:25] It exists solely for the URL space. [05:26] wgrant: Have you finished looking for DSPC usage? [05:26] You distracted me. [05:26] Heh [05:27] I suppose screaming blue murder is a bit distracting. [05:28] wgrant: how did l-s get subscribed ? [05:28] lifeless: It's the bug supervisor. [05:28] lifeless: Bug supervisors get a subscription by default, because someone likes to confuse people. [05:29] aieee [05:29] so there will be a few more to clean up [05:29] is there a page showing all l-s's subscriptions [05:29] Yes. [05:29] and the docs need tweaking [05:29] Possibly... I think it might ahave been cut. [05:29] Let me try to find it. [05:30] https://launchpad.net/~launchpad-security/+subscriptions [05:30] Yeah./ [05:30] which mingles structural and explicit. [05:30] or perhaps only shows concrete objects [05:31] StevenK: indices on non-empty-or-close-enough tables are separate patches, yes. [05:34] wgrant: cleaned up the ones i can remember making [05:34] lifeless: Thanks. [05:35] https://launchpad.net/~launchpad-security/+structural-subscriptions [05:35] and the docs are updated [05:35] Three left. [05:35] I wonder [05:35] Bug mail for Launchpad Security about Timeline is filtered; it will be sent only if it matches the following filter: [05:35] This filter allows all mail through. [05:35] Yay. [05:35] can you subscribe an arbitrary team by setting bug-supervisor [05:36] apparently I am in a team cxalled 'morfose' [05:36] You used to be able to. [05:36] I think only team members can now, though. [05:36] fixed those as well [05:36] Thankyou sir. [05:37] wgrant: DSPC, or did lifeless distract you? :-) [05:37] blargh dying [05:37]