/srv/irclogs.ubuntu.com/2013/01/31/#launchpad-dev.txt

StevenKwgrant: Thanks. When that hits devel, I'll rip out the invalidates, add one to the match method of BrowsesWithQueryLimit and toss it at ec200:00
StevenKwgrant: Yes, having just found the code in the glue, I was going to move the project back, and you alreadyd did it.01:12
wgrantYes :)01:13
wgrantloggerhead doesn't have Launchpad-specific bits01:13
wgrantThey're in launchpad_loggerhead, in the LP tree01:13
StevenKYes, the glue01:14
wgrantfailures=3601:15
StevenK37 queries/external actions issued in 4.41 seconds01:16
StevenKThat looks like SUCCESS01:16
wgrantDF?01:17
wgrantOr it on qas already?01:17
StevenKThat's qas01:17
wgrantNice01:17
wgrantubuntu needs DS preloading, but at least there's only ever likely to be a few DSes01:17
wgrantSo it's not so important.01:18
StevenKwgrant: https://code.launchpad.net/~stevenk/launchpad/loggerhead-reject-no-sreg/+merge/14576001:23
wgrantStevenK: Terrible error message is terrible.01:23
StevenKThe log, or the exception01:24
StevenK?01:24
wgrantThe one that's going to be entirely unhelpful to users :)01:24
wgrant(also, we prefer "cannot" to "can not" in most contexts)01:25
StevenKwgrant: http://pastebin.ubuntu.com/1591934/01:26
wgrantMaybe "You don't have a Launchpad account. Check that you're logged in as the right user, or log into Launchpad and try again."01:27
StevenKwgrant: I've pushed that up and the diff has updated.01:32
wgrantIt's a bit sad that you can't set explanation in __init__01:33
wgrantBut there seems to be no way around it01:33
wgrantStevenK: Is this testable?01:33
StevenKI couldn't find any tests01:33
wgrantIndeed01:33
StevenKThat entire method that block is in is mocked out01:34
wgrantr=me01:34
wgrantYeah01:34
* StevenK peers at these failures02:47
StevenKIntegrityError: new row for relation "specification" violates check constraint "specification_completion_recorded_chk"02:48
StevenKwgrant: Do you have a broken OpenID account on staging handy?02:51
wgrantStevenK: OOPS-17fcee4cc96517eb7f0dca9fe7cdf8ec03:09
wgrantcodebrowse must be lagging03:10
StevenKOh, the glue is run on tellurium?03:12
wgrantStevenK: It's run as part of the codebrowse application, yes.03:12
StevenKwgrant: No magic BFJ branches for me yet?03:23
wgrantStevenK: https://code.launchpad.net/~wgrant/launchpad/flatten-bfj-0-db/+merge/145540 https://code.launchpad.net/~wgrant/launchpad/flatten-bfj-1-populate/+merge/145541 https://code.launchpad.net/~wgrant/launchpad/flatten-bfj-2-garbo/+merge/145542 https://code.launchpad.net/~wgrant/launchpad/flatten-bfj-3-query/+merge/145543 are the majority of the current state, but not yet ready for review03:27
StevenKCrumbs03:28
wgrant0, 1, and 2 will grow a couple of extra bits each03:29
wgrant3 is basically done, as the corresponding additions are worth a 3.503:30
wgrantThere'll be a 2.5 to drop garbo03:30
StevenKYeah03:30
wgrantAnd 4 has another 100 lines of diff or so03:30
StevenKBut that's a clean revert of the revno -2 ends up as, so meh03:30
wgrantExactly03:30
StevenKwgrant: I thought our Zope machinery caught Unicode errors? Bug 89705303:37
_mup_Bug #897053: apport generated urls can trigger UnicodeDecodeError <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/897053 >03:37
wgrantStevenK: In URLs, sure03:37
wgrantThat bug's not about that.03:37
StevenKwgrant: It isn't?03:39
wgrantStevenK: That doesn't seem to be an error involving invalid Unicode in a URL segment, no03:40
StevenKOh, the URL segment is okay, it's the form data that's bong03:41
wgrantNo03:41
wgrantIt's the widget that's the problem.03:41
wgrantWell, or something's not giving the widget unicode, I guess03:42
wgrantLook at the traceback.03:42
StevenKwgrant: Haha, I see that OOPS04:01
wgrant?04:03
StevenKwgrant: https://oops.canonical.com/?oopsid=OOPS-cc84945e33282c37b36e0441a72db951 but it causes an ISE04:06
wgrantStevenK: "that OOPS"?04:08
wgrantOh04:08
StevenKwgrant: ./2013-01-31/OOPS-986af52b1270191ca6d71de17241b352 on neem, it has your username on it, and is a UnicodeDecodeError04:08
wgrantThe instance that I generated?04:08
wgrantRight04:08
StevenKI'm a little worried that it's ISEing04:09
StevenKWithout giving an OOPS id04:09
StevenKAnd the access log is not helpful04:14
wgrantThe access log won't show errors04:14
wgrantBecause it's an access log :)04:15
StevenKThe error log is also unhelpful04:15
StevenKI can view other OOPSes, just not that one04:18
StevenKwgrant: ('value', "package uml-utilities 20070815-1.1ubuntu2 failed to install/upgrade: el subproc\xe9s post-installation script retorn\xe0 el codi d'eixida d'error 1")04:32
StevenKNote the lack of u04:32
wgrantStevenK: Well, yes, that was pretty obvious from the exception :)04:32
wgrantCasting a unicode to a unicode doesn't try to decode it!04:32
StevenKStill trying to work out the cause04:34
StevenKSo it's a TextWidget, and it seems to expect unicode strings04:35
wgrantRight04:37
StevenKI wonder if it's the call to self.widgets['title'].setRenderedValue(self.extra_data.initial_summary)04:40
StevenKWhich may point to publishTraverse() not doing unicode-y things04:40
wgrantI'm not sure it's the publisher's place to decode query string arguments.04:40
wgrantWork out how it ends up undecoded.04:40
StevenKSadly, it isn't that04:41
wgrantpublishTraverse handles the path, not the query string, so I'm not terribly surprised.04:42
StevenKI mean the extra_data bit, which has a comment saying that it was populated by publishTraverse04:43
wgrantOh, are you sure you care about extra_data?04:43
wgrantThat's usually the bit retrieved from the apport blob04:44
StevenKI do not in this case04:44
wgrant(which is referenced by a trailing UUID path segment; hence the publishTraverse relevance)04:44
StevenKMy theory was incorrect04:44
StevenKH04:45
StevenK*Hmmm04:45
StevenKWe just toss the data straight into render()04:45
StevenKWCPGW and all04:46
* wgrant comes up with a cunning plan04:49
StevenKOh?04:50
wgrantbuildfarmjob will survive my purge04:50
wgrantThough it shall be gravely disfigured.04:50
StevenK/home/steven/launchpad/lp-sourcedeps/eggs/zope.app.form-3.8.1-py2.7.egg/zope/app/form/browser/textwidgets.py:139: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal04:52
StevenK  if self.convert_missing_value and input == self._missing:04:52
StevenKWhich means something is tossing a string into _toFieldValue04:53
StevenKWhich is implicated by the zope form machinery04:56
StevenKwgrant: Disfigured how?04:57
wgrantStevenK: Gravely!05:04
wgrantDuh05:04
StevenKHah05:05
StevenKwgrant: Your helpfulness knows no bounds :-P05:05
wgrant:)05:07
StevenKwgrant: In other news, I don't understand this error05:08
wgrantStevenK: Why not?05:08
StevenKIt looks like we just toss the string into the widget, and then it goes bang.05:08
wgrantWho is "we"?05:09
wgrantGiven the arg is named field.title, it's plausible that it's zope.app.form doing it itself05:09
StevenKwgrant: http://pastebin.ubuntu.com/1592257/05:10
StevenKPortion of a traceback.print_stack()05:10
wgrantStevenK: That's probably well after the problematic value has been injected05:10
StevenKwgrant: A large chunk of the traceback is all TALES, and then zope's publisher calling render()05:11
wgrantAh, that sequence isn't actually valid UTF-805:13
wgrantThat might be why it's not being automatically decoded05:13
StevenKI thought that was known05:13
StevenKI just don't know what does that part of the magic05:13
wgrantProbably zope.formlib05:16
* wgrant looks05:16
wgranthee hee05:24
wgrant        for charset in self.charsets:05:24
wgrant            try:05:24
wgrant                text = unicode(text, charset)05:24
wgrant                break05:24
wgrant            except UnicodeError:05:24
wgrant                pass05:24
StevenKOH05:25
StevenKWHY05:25
StevenKWHY05:25
StevenKIs that in zope.formlib?05:25
wgrantNope, the base BrowserRequest05:25
wgrant(which is in zope.publisher, but not actually related to the publisher)05:26
StevenKTwitch05:26
wgrantI wonder what the test suite will think of you if you override that to decode as UTF-8 or die05:26
StevenKRemove the try and except?05:26
wgrantHow did we end up handling the bad unicode in PATH_INFO thing?05:27
StevenKAt the moment, I'm wondering why Store.of(context) is None05:27
wgrantAh, that's right05:27
wgrantWe decode as UTF-8 with replace05:27
wgrantStevenK: Which?05:27
StevenK  File "/home/steven/launchpad/lp-branches/browseswithquerylimit-invalidates/lib/lp/testing/matchers.py", line 84, in match05:28
StevenK    Store.of(context).invalidate()05:28
wgrantStevenK: This decoding happens later, so you might be able to raise an exception that you can turn into a 40005:28
StevenKAttributeError: 'NoneType' object has no attribute 'invalidate'05:28
wgrant(we couldn't do that with PATH_INFO because it was too early)05:28
wgrantWhat is context?05:28
StevenKwgrant: So inside zope.publisher, or we call into the publisher?05:29
StevenK(Pdb) p context05:30
StevenK<lp.registry.model.distributionsourcepackage.DistributionSourcePackage object at 0x10887fd0>05:30
wgrantStevenK: DSPs aren't real objects05:30
wgrantTry DSP.distribution05:30
wgrant(the distributionsourcepackage table is represented by DistributionSourcePackageInDatabase, not DistributionSourcePackage)05:30
wgrantStevenK: I'd override LaunchpadRequest._decode or so05:30
wgrantEither make it decode('utf-8', 'replace') or catch the UnicodeDecodeError and give a 40005:31
StevenK(Pdb) p context05:31
StevenK<lp.registry.model.milestone.ProjectMilestone object at 0xf0a6dd0>05:31
wgrant'cause if someone uses anything other than UTF-8 then they're not worth our time :)05:31
wgrantProjectMilestone also isn't real05:31
wgrantCould just invalidate all stores05:31
StevenKThat would work too05:31
StevenKIStore.invalidate() ?05:32
wgrantNo05:32
wgrantInterfaces don't work that way05:32
wgrantlp.services.database.sqlbase.flush_database_caches()05:32
StevenKOh, nice. That branch failed to scan six hours ago, and I didn't notice.05:35
StevenKwgrant: You mean LaunchpadBrowserRequest ?05:36
StevenKThe interface seems to be IBasicLaunchpadRequest05:37
wgrantStevenK: Possibly, or something lower than that05:37
wgrantI forget what the hierarchy looks like05:37
StevenKI can't find a LaunchpadRequest class, that's all05:37
wgrantBut any of our request classes that inherit zope.publisher.browser.BrowserRequest at all want it05:37
wgrantTHey might all inherit LaunchpadBrowserRequest05:38
StevenKclass LaunchpadBrowserRequestMixin:05:39
StevenK    """Provides methods used for both API and web browser requests."""05:39
StevenKHaha05:39
StevenKwgrant: lp.services.webapp.servers.get_query_string_params05:40
wgrantStevenK: Yeah, that uses the same _decode method05:47
StevenKwgrant: https://code.launchpad.net/~stevenk/launchpad/browseswithquerylimit-invalidates/+merge/14577405:49
wgrantStevenK: You'll want to flush before you register the query collector, otherwise any pending writes will get collected05:52
wgrantAlso, test_sprint has a flush that's now redundant05:52
StevenKwgrant: http://pastebin.ubuntu.com/1592333/05:56
wgrantStevenK: Seems odd to split the collector definition and registration, but otherwise fine05:57
StevenKwgrant: http://pastebin.ubuntu.com/1592336/05:57
wgrantIndeed05:57
StevenKwgrant: That MP is now with more red and a little more green06:04
wgrantStevenK: r=me, thanks06:05
StevenKdb-devel is obviously cursed06:09
wgrantApparently.06:10
=== yofel_ is now known as yofel
adeuringgood morning09:04
=== slank_away is now known as slank
=== matsubara is now known as matsubara-lunch
=== matsubara-lunch is now known as matsubara
=== deryck is now known as deryck[lunch]
=== deryck[lunch] is now known as deryck
=== matsubara is now known as matsubara-afk

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!