[00:01] wgrant: [00:01] (Pdb) p hasattr(item, 'file') [00:01] True [00:01] (Pdb) p hasattr(item, 'filename') [00:01] True [00:01] StevenK: Is the filename non-empty? [00:01] Nope [00:01] It's None [00:02] Aha [00:02] So [00:02] Compare the requests that are sent [00:09] I wonder about lazr/restful/_bytestorage.py [00:11] StevenK: That's a Resource [00:11] Not relevant here [00:12] (though you should probably also test it, it's probably fine as it will just use the raw body) === slank is now known as slank_away [02:55] BLEH [02:56] The traceback does not show lazr.restful's call [02:56] Just the WSGI and the zope publisher [02:57] Which traceback? [02:58] The pdb I have in zope.publisher.browser [02:58] I was hoping to get a clue what called it [02:58] That's before lazr.restful, usually [02:58] Oh, then what forms this request? [02:59] StevenK: lazr.restful interprets the request [03:00] After it gets through the normal Zope request stuff, which eventually decides that it's an API request [03:01] So it could be launchpadlib itself, then? [03:03] StevenK: Right, I'd examine the POST request it makes [03:03] Using httplib2.debuglevel === tumbleweed_ is now known as tumbleweed [03:55] httplib2.debuglevel seems to be working great. Not. [03:56] StevenK: You need to set it before you instantiate the launchpad object [03:56] Oh [03:57] wgrant: http://pastebin.ubuntu.com/1611158/ [03:57] That looks wrong already [03:59] Yeah, there's the problem [04:00] So it could be lazr.restfulclient? [04:01] It is, yes. [04:01] So we ideally want to fix it and work around on the server side too [04:01] Although I'm not quite sure what lazr.restfulclient can do [04:01] It doesn't necessarily have a filename [04:02] It may be better to see if addReleaseFile can somehow obtain the request, and get the file out somewhat manually [04:02] Or we could make lazr.restful do that [04:03] I was thinking we could change lazr.restfulclient to set a filename of 'bytestream' or something like that [04:04] We have to do something server-side anyway [04:04] What are you thinking? [04:06] We need to either fix lazr.restfulclient to send the header and fix the server to reject the request if the content is a unicode, thus making everyone upgrade... or we can fix lazr.restfulclient for the future and make the server grab the str directly from the request, bypassing the decoded value [04:09] Sounds like you prefer the second option. [04:09] We want to avoid breaking clients if we can. [04:12] wgrant: Does that bug have enough branches linked to it? [04:13] No [04:13] I could link the 5-10 I landed before I started linking things if you want :) [04:13] Haha [04:14] wgrant: QA and deployment, or what is your plan? [04:15] StevenK: Hahahaa no deploying now would be suicide [04:15] There is still stuff to land? [04:15] There's about a dozen indices to go [04:15] Working through the last few pages now [04:16] Right [04:18] Hmm, now. How to get at the request [04:28] (Pdb) p request [04:28] [04:30] did you use get_current_browser_request()? [04:31] Yeah [04:31] Still trying to figure out what I want to pull out from it [04:31] Yeah, I'm not quite sure [04:32] Look at how BrowserRequest.__processInput gets stuff [04:32] That iterates over items() [04:32] I think [04:32] But items() for these LTRs is not helpful [04:33] Or it might no longer be helpful [04:33] It's possible that it gets consumed [04:34] It looks to play with self._environ, and then throws that to ZopeFieldStorage [04:34] request._environ has stuff in it, but nothing of any use [04:35] It doesn't use request.read()? [04:35] The FieldStorage itself might do that [04:35] *** AttributeError: AttributeError("'LaunchpadTestRequest' object has no attribute 'read'",) [04:36] Well, it's something like that [04:37] Maybe request.body.read() [04:37] Anyway, can you obtain the ZFS? [04:38] I don't think the ZFS is saved anywhere [04:38] So I think the answer is no [04:39] :( [04:40] Also, if I was to come up with a list of good ideas, reading the file into RAM would not appear on it [04:40] Bah, RAM is cheap [04:40] :-P [04:55] wgrant: http://pastebin.ubuntu.com/1611231/ not sure if the request is actually helpful [04:57] StevenK: The environment won't be useful for the request body, no [04:57] But that's what ZFS is pulling from [04:58] Oh [04:58] There's extra stuff in there [04:58] wsgi.input [04:58] (Pdb) p request.form['field.filecontent'] [04:58] [04:59] That sounds like it was correctly submitted, and to +addownloadfile, not the API [04:59] Which it was, yes [05:01] (Pdb) p request.form['file_content'] [05:01] u'\x1f\...' [05:01] Right, broken like hell [05:03] wgrant: It's broken in request.items() too [05:03] Naturally. [05:03] that just iterates over the form items, doesn't it? [05:03] Not the raw request data [05:04] Then where would that be hiding? [05:05] In _environ. I assume ZFS parses that [05:05] (Pdb) p request._environ['wsgi.input'].read() [05:05] '' [05:06] Not so helpful [05:06] seek(0) [05:08] Content-Disposition: form-data; name="file_content"\n\n\x1f... [05:08] Right :) [05:08] You might be able to use ZopeFieldStorage to parse it more easily and get the raw content without parsing the MIME yourself [05:08] So it's broken everywhere? [05:09] Hmm? [05:09] But that is from the MIME? [05:09] What's wrong with it? [05:09] And it looks encoded already [05:09] It's a bytestring [05:09] Which is what we want [05:09] \x1f isn't an obvious test, since it's ASCII. [05:09] So decoding it as UTF-8 won't break [05:09] But that looks fine to me [05:10] name="file_content"\n\n\x1f\x8b\x08\x00\xcc\x03\x0fQ\x00\x03\xed\xce1\x12\x82@ [05:10] Still looks okay? [05:10] Nothing in there looks like a U+FFFD [05:10] So indeed [05:10] Compare it to the data you gave it? [05:11] Don't know how to get it in that format [05:11] Hmm? [05:11] What format? [05:11] That's just the raw bytes [05:11] (but \x-escaped because you're repr()ing the str) [05:12] It's a tarball, so ... [05:12] open('foo.tar').read()[:20] [05:13] Ah [05:13] It is identical [05:13] Good :) [05:14] It would be a matter of great concern if it did not [05:14] Heh [05:14] So I want to rely on _environ['wsgi.input'] ? Seems like an awful hack [05:14] It is a terrible revolting hack [05:15] But if Zope doesn't give us another way to get it, then we have little choice [05:22] ZFS is just cgi.FieldStorage [05:22] ... which is nicely undocumented === cyclicflux_ is now known as CyclicFlux [05:48] * StevenK stabs cgi.FieldStorage [05:57] wgrant: http://pastebin.ubuntu.com/1611421/ [06:03] StevenK: Isn't that unicodeness check around the wrong way? [06:03] We want to reretrieve it if *has* been decoded [06:03] (we also want to run this past gary_poster or benji) [06:04] Yeah [06:04] Fixed [06:05] Also, that's really awful and I hope you pay for your crimes. [06:05] But it looks quite effective [06:05] Hahaha [06:05] I really don't want to use it [06:06] wgrant: The other half of that branch is the _decode resurrection [06:07] Right [08:52] good morning [10:17] http://webnumbr.com/launchpad-critical-bugs is looking pretty good [10:19] not bad for only having two people on there :) [10:23] heh [10:24] still more evidence against using centralized allocation of scarce resources to meet inexhaustible demand? [10:25] wgrant: StevenK if either of ye are about I'd apprecite some help on answers trying to catch up from being off and some of them need yer help [10:25] please [10:26] https://bugs.launchpad.net/launchpad/+bug/1108790 bug or not a bug also [10:26] <_mup_> Bug #1108790: Ubuntu-launchpad uses google-analytics by default < https://launchpad.net/bugs/1108790 > [10:27] czajkowski: well it has flattened off recently [10:28] czajkowski: but yeah [10:29] jml: possibly eveidence that changing things when the system is unstable leads to more instability [10:29] lifeless: heh, maybe [10:29] lifeless: Hmmm? [10:30] lifeless: We know from the lpstats graphs that the number of incoming bugs did not change significantly [10:30] We're just less hopeless at closing them [10:30] wgrant: presumably all the low hanging fruit :) [10:31] wgrant: have you repeated the root cause analysis flacoste did? [10:31] lifeless: No [10:31] wgrant: that was fairly conclusive that most criticals were self inflicted :) [10:31] IIRC === Ursinha is now known as Ursinha-afk [10:51] wgrant: how long is buildbot now? [10:53] lifeless: 38 minutesish === Ursinha-afk is now known as Ursinha [11:01] hmm I still need to write up post fosdem review [11:01] can I have another 4 hours in todays day please [11:09] czajkowski: Sure, if you have four hours less sleep. [11:10] I am already running on low from the weekend of brussels, beer and geeks = late night chats and catch up [11:10] was great fun [11:10] though I thnk I've had enough waffles to last me till next year === Ursinha is now known as Ursinha-afk === Ursinha-afk is now known as Ursinha [11:13] czajkowski: amen :) [11:14] heh [11:14] it was very good this year , lifeless you ever been? [11:15] czajkowski: no, I haven't. I was at LCA though - same week :) [11:16] ah not been there, this is a little closer for me to get to [11:17] although there was a large USA attendees who all flew in for the weekend [11:17] plus RMS arrived [11:17] alhtough he was banned/asked not to attend the legal sessions [11:20] ROTFL [11:20] I probably shouldn't be so amused. [11:20] But still, I am. [11:21] so was I when I heard [11:21] there was much running around keeping him away from perople it seems [11:21] he's now in ireland [11:21] man on a mission [11:22] to aleniate a lot of people [11:28] wgrant: Did you get a chance to look at bpph-phase? === Ursinha is now known as Ursinha-afk === Ursinha-afk is now known as Ursinha === Ursinha is now known as Ursinha-afk === Ursinha-afk is now known as Ursinha === yofel_ is now known as yofel === slank_away is now known as slank === deryck is now known as deryck[lunch] === slank is now known as slank_away === slank_away is now known as slank === deryck[lunch] is now known as deryck === slank is now known as slank_away === slank_away is now known as slank [23:46] wgrant: https://code.launchpad.net/~stevenk/launchpad/unexport-updatepreviewdiff/+merge/146744 [23:53] StevenK: https://code.launchpad.net/~wgrant/launchpad/setupCompleteBuilds-nonspecific/+merge/146751 [23:54] StevenK: r=me [23:54] wgrant: Can't getSpecificJobs die as pointless now? [23:55] StevenK: No [23:55] StevenK: You need to get the specific jobs for Archive:+builds and Builder:+history [23:55] Because the underlying queries just give you BuildFarmJobs, so you need to translate them. [23:56] Right [23:57] wgrant: r=me [23:57] StevenK: Thanks [23:57] And Firefox hangs for 30 seconds because I dared to close a tab that contained Flash === slank is now known as slank_away