[00:01] <StevenK> wgrant:
[00:01] <StevenK> (Pdb) p hasattr(item, 'file')
[00:01] <StevenK> True
[00:01] <StevenK> (Pdb) p hasattr(item, 'filename')
[00:01] <StevenK> True
[00:01] <wgrant> StevenK: Is the filename non-empty?
[00:01] <StevenK> Nope
[00:01] <StevenK> It's None
[00:02] <wgrant> Aha
[00:02] <wgrant> So
[00:02] <wgrant> Compare the requests that are sent
[00:09] <StevenK> I wonder about lazr/restful/_bytestorage.py
[00:11] <wgrant> StevenK: That's a Resource
[00:11] <wgrant> Not relevant here
[00:12] <wgrant> (though you should probably also test it, it's probably fine as it will just use the raw body)
[02:55] <StevenK> BLEH
[02:56] <StevenK> The traceback does not show lazr.restful's call
[02:56] <StevenK> Just the WSGI and the zope publisher
[02:57] <wgrant> Which traceback?
[02:58] <StevenK> The pdb I have in zope.publisher.browser
[02:58] <StevenK> I was hoping to get a clue what called it
[02:58] <wgrant> That's before lazr.restful, usually
[02:58] <StevenK> Oh, then what forms this request?
[02:59] <wgrant> StevenK: lazr.restful interprets the request
[03:00] <wgrant> After it gets through the normal Zope request stuff, which eventually decides that it's an API request
[03:01] <StevenK> So it could be launchpadlib itself, then?
[03:03] <wgrant> StevenK: Right, I'd examine the POST request it makes
[03:03] <wgrant> Using httplib2.debuglevel
[03:55] <StevenK> httplib2.debuglevel seems to be working great. Not.
[03:56] <wgrant> StevenK: You need to set it before you instantiate the launchpad object
[03:56] <StevenK> Oh
[03:57] <StevenK> wgrant: http://pastebin.ubuntu.com/1611158/
[03:57] <StevenK> That looks wrong already
[03:59] <wgrant> Yeah, there's the problem
[04:00] <StevenK> So it could be lazr.restfulclient?
[04:01] <wgrant> It is, yes.
[04:01] <wgrant> So we ideally want to fix it and work around on the server side too
[04:01] <wgrant> Although I'm not quite sure what lazr.restfulclient can do
[04:01] <wgrant> It doesn't necessarily have a filename
[04:02] <wgrant> It may be better to see if addReleaseFile can somehow obtain the request, and get the file out somewhat manually
[04:02] <wgrant> Or we could make lazr.restful do that
[04:03] <StevenK> I was thinking we could change lazr.restfulclient to set a filename of 'bytestream' or something like that
[04:04] <wgrant> We have to do something server-side anyway
[04:04] <StevenK> What are you thinking?
[04:06] <wgrant> We need to either fix lazr.restfulclient to send the header and fix the server to reject the request if the content is a unicode, thus making everyone upgrade... or we can fix lazr.restfulclient for the future and make the server grab the str directly from the request, bypassing the decoded value
[04:09] <StevenK> Sounds like you prefer the second option.
[04:09] <wgrant> We want to avoid breaking clients if we can.
[04:12] <StevenK> wgrant: Does that bug have enough branches linked to it?
[04:13] <wgrant> No
[04:13] <wgrant> I could link the 5-10 I landed before I started linking things if you want :)
[04:13] <StevenK> Haha
[04:14] <StevenK> wgrant: QA and deployment, or what is your plan?
[04:15] <wgrant> StevenK: Hahahaa no deploying now would be suicide
[04:15] <StevenK> There is still stuff to land?
[04:15] <wgrant> There's about a dozen indices to go
[04:15] <wgrant> Working through the last few pages now
[04:16] <StevenK> Right
[04:18] <StevenK> Hmm, now. How to get at the request
[04:28] <StevenK> (Pdb) p request
[04:28] <StevenK> <lp.services.webapp.servers.LaunchpadTestRequest instance URL=http://launchpad.dev>
[04:30] <wgrant> did you use get_current_browser_request()?
[04:31] <StevenK> Yeah
[04:31] <StevenK> Still trying to figure out what I want to pull out from it
[04:31] <wgrant> Yeah, I'm not quite sure
[04:32] <wgrant> Look at how BrowserRequest.__processInput gets stuff
[04:32] <StevenK> That iterates over items()
[04:32] <StevenK> I think
[04:32] <StevenK> But items() for these LTRs is not helpful
[04:33] <wgrant> Or it might no longer be helpful
[04:33] <wgrant> It's possible that it gets consumed
[04:34] <StevenK> It looks to play with self._environ, and then throws that to ZopeFieldStorage
[04:34] <StevenK> request._environ has stuff in it, but nothing of any use
[04:35] <wgrant> It doesn't use request.read()?
[04:35] <wgrant> The FieldStorage itself might do that
[04:35] <StevenK> *** AttributeError: AttributeError("'LaunchpadTestRequest' object has no attribute 'read'",)
[04:36] <wgrant> Well, it's something like that
[04:37] <wgrant> Maybe request.body.read()
[04:37] <wgrant> Anyway, can you obtain the ZFS?
[04:38] <StevenK> I don't think the ZFS is saved anywhere
[04:38] <StevenK> So I think the answer is no
[04:39] <wgrant> :(
[04:40] <wgrant> Also, if I was to come up with a list of good ideas, reading the file into RAM would not appear on it
[04:40] <StevenK> Bah, RAM is cheap
[04:40] <StevenK> :-P
[04:55] <StevenK> wgrant: http://pastebin.ubuntu.com/1611231/ not sure if the request is actually helpful
[04:57] <wgrant> StevenK: The environment won't be useful for the request body, no
[04:57] <StevenK> But that's what ZFS is pulling from
[04:58] <wgrant> Oh
[04:58] <wgrant> There's extra stuff in there
[04:58] <wgrant> wsgi.input
[04:58] <StevenK> (Pdb) p request.form['field.filecontent']
[04:58] <StevenK> <zope.publisher.browser.FileUpload object at 0x2abb382f71d0>
[04:59] <wgrant> That sounds like it was correctly submitted, and to +addownloadfile, not the API
[04:59] <StevenK> Which it was, yes
[05:01] <StevenK> (Pdb) p request.form['file_content']
[05:01] <StevenK> u'\x1f\...'
[05:01] <StevenK> Right, broken like hell
[05:03] <StevenK> wgrant: It's broken in request.items() too
[05:03] <wgrant> Naturally.
[05:03] <wgrant> that just iterates over the form items, doesn't it?
[05:03] <wgrant> Not the raw request data
[05:04] <StevenK> Then where would that be hiding?
[05:05] <wgrant> In _environ. I assume ZFS parses that
[05:05] <StevenK> (Pdb) p request._environ['wsgi.input'].read()
[05:05] <StevenK> ''
[05:06] <StevenK> Not so helpful
[05:06] <wgrant> seek(0)
[05:08] <StevenK> Content-Disposition: form-data; name="file_content"\n\n\x1f...
[05:08] <wgrant> Right :)
[05:08] <wgrant> You might be able to use ZopeFieldStorage to parse it more easily and get the raw content without parsing the MIME yourself
[05:08] <StevenK> So it's broken everywhere?
[05:09] <wgrant> Hmm?
[05:09] <StevenK> But that is from the MIME?
[05:09] <wgrant> What's wrong with it?
[05:09] <StevenK> And it looks encoded already
[05:09] <wgrant> It's a bytestring
[05:09] <wgrant> Which is what we want
[05:09] <wgrant> \x1f isn't an obvious test, since it's ASCII.
[05:09] <wgrant> So decoding it as UTF-8 won't break
[05:09] <wgrant> But that looks fine to me
[05:10] <StevenK> name="file_content"\n\n\x1f\x8b\x08\x00\xcc\x03\x0fQ\x00\x03\xed\xce1\x12\x82@
[05:10] <StevenK> Still looks okay?
[05:10] <wgrant> Nothing in there looks like a U+FFFD
[05:10] <wgrant> So indeed
[05:10] <wgrant> Compare it to the data you gave it?
[05:11] <StevenK> Don't know how to get it in that format
[05:11] <wgrant> Hmm?
[05:11] <wgrant> What format?
[05:11] <wgrant> That's just the raw bytes
[05:11] <wgrant> (but \x-escaped because you're repr()ing the str)
[05:12] <StevenK> It's a tarball, so ...
[05:12] <wgrant> open('foo.tar').read()[:20]
[05:13] <StevenK> Ah
[05:13] <StevenK> It is identical
[05:13] <wgrant> Good :)
[05:14] <wgrant> It would be a matter of great concern if it did not
[05:14] <StevenK> Heh
[05:14] <StevenK> So I want to rely on _environ['wsgi.input'] ? Seems like an awful hack
[05:14] <wgrant> It is a terrible revolting hack
[05:15] <wgrant> But if Zope doesn't give us another way to get it, then we have little choice
[05:22] <StevenK> ZFS is just cgi.FieldStorage
[05:22] <StevenK> ... which is nicely undocumented
[05:48]  * StevenK stabs cgi.FieldStorage
[05:57] <StevenK> wgrant: http://pastebin.ubuntu.com/1611421/
[06:03] <wgrant> StevenK: Isn't that unicodeness check around the wrong way?
[06:03] <wgrant> We want to reretrieve it if *has* been decoded
[06:03] <wgrant> (we also want to run this past gary_poster or benji)
[06:04] <StevenK> Yeah
[06:04] <StevenK> Fixed
[06:05] <wgrant> Also, that's really awful and I hope you pay for your crimes.
[06:05] <wgrant> But it looks quite effective
[06:05] <StevenK> Hahaha
[06:05] <StevenK> I really don't want to use it
[06:06] <StevenK> wgrant: The other half of that branch is the _decode resurrection
[06:07] <wgrant> Right
[08:52] <adeuring> good morning
[10:17] <lifeless> http://webnumbr.com/launchpad-critical-bugs﻿ is looking pretty good
[10:19] <czajkowski> not bad for only having two people on there :)
[10:23] <jml> heh
[10:24] <jml> still more evidence against using centralized allocation of scarce resources to meet inexhaustible demand?
[10:25] <czajkowski> wgrant: StevenK if either of ye are about I'd apprecite some help on answers trying to catch up from being off and some of them need yer help
[10:25] <czajkowski> please
[10:26] <czajkowski> https://bugs.launchpad.net/launchpad/+bug/1108790  bug or not a bug also
[10:26] <_mup_> Bug #1108790: Ubuntu-launchpad uses google-analytics by default <Launchpad itself:New> < https://launchpad.net/bugs/1108790 >
[10:27] <lifeless> czajkowski: well it has flattened off recently
[10:28] <lifeless> czajkowski: but yeah
[10:29] <lifeless> jml: possibly eveidence that changing things when the system is unstable leads to more instability
[10:29] <jml> lifeless: heh, maybe
[10:29] <wgrant> lifeless: Hmmm?
[10:30] <wgrant> lifeless: We know from the lpstats graphs that the number of incoming bugs did not change significantly
[10:30] <wgrant> We're just less hopeless at closing them
[10:30] <lifeless> wgrant: presumably all the low hanging fruit :)
[10:31] <lifeless> wgrant: have you repeated the root cause analysis flacoste did?
[10:31] <wgrant> lifeless: No
[10:31] <lifeless> wgrant: that was fairly conclusive that most criticals were self inflicted :)
[10:31] <lifeless> IIRC
[10:51] <lifeless> wgrant: how long is buildbot now?
[10:53] <wgrant> lifeless: 38 minutesish
[11:01] <czajkowski> hmm I still need to write up post fosdem review
[11:01] <czajkowski> can I have another 4 hours in todays day please
[11:09] <StevenK> czajkowski: Sure, if you have four hours less sleep.
[11:10] <czajkowski> I am already running on low from the weekend of brussels, beer and geeks = late night chats and catch up
[11:10] <czajkowski> was great fun
[11:10] <czajkowski> though I thnk I've had enough waffles to last me till next year
[11:13] <lifeless> czajkowski: amen :)
[11:14] <czajkowski> heh
[11:14] <czajkowski> it was very good this year , lifeless you ever been?
[11:15] <lifeless> czajkowski: no, I haven't. I was at LCA though - same week :)
[11:16] <czajkowski> ah not been there, this is a little closer for me to get to
[11:17] <czajkowski> although there was a large USA attendees who all flew in for the weekend
[11:17] <czajkowski> plus RMS arrived
[11:17] <czajkowski> alhtough he was banned/asked not to attend the legal sessions
[11:20] <lifeless> ROTFL
[11:20] <lifeless> I probably shouldn't be so amused.
[11:20] <lifeless> But still, I am.
[11:21] <czajkowski> so was I when I heard
[11:21] <czajkowski> there was much running around keeping him away from perople it seems
[11:21] <czajkowski> he's now in ireland
[11:21] <lifeless> man on a mission
[11:22] <czajkowski> to aleniate a lot of people
[11:28] <cjwatson> wgrant: Did you get a chance to look at bpph-phase?
[23:46] <StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/unexport-updatepreviewdiff/+merge/146744
[23:53] <wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/setupCompleteBuilds-nonspecific/+merge/146751
[23:54] <wgrant> StevenK: r=me
[23:54] <StevenK> wgrant: Can't getSpecificJobs die as pointless now?
[23:55] <wgrant> StevenK: No
[23:55] <wgrant> StevenK: You need to get the specific jobs for Archive:+builds and Builder:+history
[23:55] <wgrant> Because the underlying queries just give you BuildFarmJobs, so you need to translate them.
[23:56] <StevenK> Right
[23:57] <StevenK> wgrant: r=me
[23:57] <wgrant> StevenK: Thanks
[23:57] <StevenK> And Firefox hangs for 30 seconds because I dared to close a tab that contained Flash