| spiv | poolie: I do get the sense that either the subunit protocol or the implementation of it has awful failure modes | 00:01 |
|---|---|---|
| poolie | yeah basicalyl that | 00:01 |
| wgz | ...I just realised I tyoped 'Pageant' in the thread title on the bazaar list | 00:01 |
| poolie | jelmer, i'm getting a failure in | 00:01 |
| poolie | ERROR: bzrlib.tests.per_tree.test_inv.TestInventory.test_canonical_invalid_child(GitWorkingTreeFormat) | 00:01 |
| poolie | with tip of bzr-git | 00:02 |
| poolie | do you want me to report it? | 00:02 |
| poolie | (other than on irc :) | 00:02 |
| spiv | poolie: I suppose if you were to design a protocol to be reasonably robust about failing well in the case of corrupted/interleaved output you'd want somethign like "length-prefixed chunks with some sort of redundant check bytes on the prefix" | 00:02 |
| poolie | i think it is in a bit of an uncanny valley between "tolerates random insertion and modification and picks out what structure it can" and "binary protocol must be passed byte-for-byte" | 00:03 |
| poolie | some parts are one and some the other | 00:03 |
| poolie | and in failure modes you get kind of the worst of both worlds | 00:03 |
| spiv | Or even just a chunk sequence number (which would mitigate against being interleaved with another valid subunit stream, if that's a realistic concern?) | 00:03 |
| jelmer | poolie: there's still about 500 failures like that in bzr-git, because roundtripping and various other bits aren't complete | 00:04 |
| jelmer | poolie: so you're welcome to file a bugreport, but I should get to that issue eventually without one too | 00:04 |
| spiv | Ugh, yes, that sounds like a taking all the downsides of Postel's Law without many of the upsides! | 00:04 |
| jelmer | poolie: for bzr-svn it's different, if you see failures like this with that then I'm very interested in bug reports | 00:05 |
| poolie | spiv yeah it is inconsistently liberal | 00:05 |
| poolie | jelmer, ok, i was a bit overoptimistic and thought i'd run my tests without bzr-git skipped | 00:06 |
| wgz | it's basically impossible to write good ctypes code | 00:27 |
| wgz | I try a different style every time I need to do something, and have never come across one I like. | 00:27 |
| === glyph_ is now known as glyph | ||
| thomi|work | wgz: I just tried that debug version of pageant, and it fixes the issue. Are you interested in it's output at all? Any chance you could provide a version that doesn't show the debug info? | 01:28 |
| poolie | hi thomi|work | 01:40 |
| thomi|work | hi poolie | 01:40 |
| Noldorin | poolie, well, any thoughts? | 03:19 |
| poolie | Noldorin, i have many thoughts | 03:24 |
| poolie | but if you asked a question, i didn't see it | 03:24 |
| Noldorin | poolie, heh yes, i'm sure you do. i did. should have your name highlighted above :-) | 03:35 |
| Noldorin | if you have a log... | 03:35 |
| poolie | oh about download types? | 03:39 |
| poolie | oh i didn't realize that was you, hi | 03:39 |
| Noldorin | no prob, hi :-) | 03:39 |
| poolie | so, let's see | 03:41 |
| poolie | actually, let's go to #launchpad-dev? | 03:41 |
| poolie | Noldorin, anyhow | 04:04 |
| poolie | there are two things i can think of to do first | 04:04 |
| poolie | one is help you get lp running locally | 04:04 |
| poolie | the other is to talk about how this change could be put in | 04:04 |
| Noldorin | poolie, sorry, we can go there if you like | 04:05 |
| vila | hi all ! | 07:30 |
| fullermd | Geez, that's cheerful. Must be post-coffee... | 07:30 |
| vila | :) | 07:33 |
| poolie | hi vila | 07:34 |
| mgz | morning! | 08:58 |
| vila | mgz: hey ! otp right now, but I'd be happy to discuss bug #892992 later | 09:12 |
| ubot5 | Launchpad bug 892992 in Bazaar "bzrlib.tests.test_config.TestStackCrossSectionsExpand.test_cross_related_sections failure" [High,Confirmed] https://launchpad.net/bugs/892992 | 09:12 |
| mgz | vila: yeah, just saw that in bugmail | 09:17 |
| poolie | o/ mgz | 09:23 |
| poolie | vila, so bzr remerge is a bit dangerous | 09:25 |
| * vila nods | 09:25 | |
| poolie | perhaps it should default to only doing conflicted files | 09:25 |
| poolie | or maybe only named files | 09:25 |
| vila | yup | 09:25 |
| vila | I think it didn't because we have cases where the same file where involved in several conflicts or some conflicts may be related and soring them out was hard | 09:26 |
| vila | afaik, I've fixed all cases where the same file were involved in several conflicts so it may be possible to be do it right now | 09:27 |
| poolie | hm | 09:27 |
| poolie | so that could be done separately, and probably should be | 09:27 |
| vila | otherwise, that will be a limitation requiring that all other conflicts (not .po related) should resolved first | 09:27 |
| poolie | ok | 09:30 |
| poolie | so that sounds pretty good | 09:30 |
| poolie | you perhaps should explain in more detail on the bug how the proposed fix will work | 09:30 |
| poolie | so slangasek is clear | 09:31 |
| vila | yup | 09:32 |
| poolie | hm | 09:34 |
| poolie | the other thing we really need which i have saliently failed to do yet | 09:34 |
| poolie | is a restfulclient update | 09:34 |
| poolie | every time lp does a fastdowntime it bumps the importer | 09:35 |
| poolie | mostly avoidably | 09:35 |
| vila | right, I've recorded the corresponding failures so far. | 09:35 |
| vila | Some are related to restfulclient, but not all. | 09:36 |
| vila | So having a way to list/remove them still sounds like a good idea, you could get the restfulclient ones from that | 09:37 |
| poolie | what do you mean? | 09:38 |
| poolie | vila, i would love if bzr gardner worked in a colo | 09:39 |
| vila | it should and will ;) | 09:40 |
| vila | err, it probably doesn't yet is what I meant | 09:40 |
| poolie | i understand | 09:41 |
| poolie | hi mgz, how's it going? | 10:02 |
| mgz | hey poolie, well. | 10:07 |
| poolie | i was going to chat with you but it's getting pretty late | 10:08 |
| mgz | we didn't settle an exact time to move our 1-1 to I don't think, maybe we should do that at least now | 10:09 |
| poolie | mgz, vila, jelmer, i was wondering the other day what to do at the rally | 10:34 |
| poolie | maybe bfbia | 10:34 |
| poolie | maybe showing more branch content in the lp ui | 10:34 |
| poolie | maybe we should also have a more specifically bzr focussed event | 10:34 |
| mgz | I thought it'd be a good chance to look at some of the udd workflow/quilt things | 10:35 |
| poolie | oh, yeah, that would be a really nice thing to work on too | 10:35 |
| poolie | especially with all the platform people | 10:35 |
| vila | +1 on reserving at least part of the event to be bzr-focused | 10:36 |
| mgz | now I've worked out what bfbia is, that's also what you were saying :) | 10:36 |
| poolie | mm | 10:36 |
| poolie | that's a bit more specific | 10:36 |
| vila | otherwise, getting launchpad running in an lxc container whould be very helpful to help debug the importer | 10:36 |
| poolie | mm | 10:37 |
| poolie | does it have to be lxc? | 10:37 |
| poolie | it's pretty trivial to get it going in a schroot | 10:37 |
| poolie | you know schroots :) | 10:37 |
| poolie | and you avoid some amount of lxc flakiness | 10:37 |
| vila | right, but lp declares a bunch a specific hosts too | 10:37 |
| vila | and not polluting the host (as opposed to guest, gee synonyms) will make it easier to manage | 10:42 |
| poolie | hm? | 10:55 |
| poolie | you're talking about putting them into /etc/hosts? | 10:55 |
| poolie | you could run the udd importer from a separate schroot | 10:56 |
| poolie | then the real host won't need to know the name of anything | 10:56 |
| * vila blinks | 10:58 | |
| vila | I should know that /etc/hosts is part of the chroot don't I ? | 10:58 |
| mgz | ah, I missed thomi|work after I went to bed last night | 11:34 |
| mgz | thomi|work: I am a little interested in the output, and have forwarded the fix upstream (other changes on trunk may also have avoided the bug for you) | 11:35 |
| mgz | I'll put up a build without the debug output later, but hopefully putty will do a new release relatively soon too | 11:37 |
| jelmer | mgz: wrt our discussion yesterday, would it perhaps make sense to make the iter_revisions / iter_files_bytes HPSS calls take a compression ("bz2", "zlib") argument? | 12:05 |
| spiv | jelmer: sounds ok to me (although I'd be a touch concerned about the load that might add to a server) | 12:07 |
| jelmer | hi spiv | 12:08 |
| jelmer | spiv: do you know why get_parent_map uses bz2 rather than anything else? | 12:08 |
| spiv | jelmer: in the most ideal of worlds the server would be able to send that those records directly from the repository's internal storage format already compressed without needing to do any processing at all | 12:09 |
| jelmer | spiv: ah, I see | 12:09 |
| spiv | (perhaps not for iter_files_bytes, but iter_revisions in principle should just be sending a record stream) | 12:09 |
| spiv | jelmer: get_parent_map uses bz2 because that's what lifeless chose to make it do :) | 12:09 |
| spiv | jelmer: so ask him :) | 12:10 |
| jelmer | spiv: will do, thanks :) | 12:10 |
| spiv | (I suspect the answer is something like: that for the relatively small amounts of data involved the CPU load on the server probably isn't a big deal vs the savings in round-trips and thus making the server repeat work re-reading indices) | 12:11 |
| spiv | But lifeless being lifeless I expect he measured :) | 12:11 |
| spiv | jelmer: hmm | 12:12 |
| mgz | spiv: that ideal is the same I was wishing after last night :) | 12:12 |
| spiv | jelmer: so iter_files_bytes could also be built on record streams | 12:12 |
| jelmer | spiv: the server side implementation already relies on get_record_stream | 12:12 |
| spiv | jelmer: and the record stream APIs already have a way to express “hey I'd prefer stuff optimised for repo format X” | 12:12 |
| spiv | jelmer: but the limitation you'll need to work around is that at the moment the queries to generate record streams aren't very flexible: | 12:13 |
| mgz | I'm reasonable sure bz2 was smaller when it was added, but I'm curious how much that's still true, the cost is pretty high | 12:13 |
| spiv | jelmer: it's easy to say “give me all revisions+inventories+texts+signatures+chks for {list of revisions, full ancestry of list of revisions}" | 12:14 |
| spiv | jelmer: but atm it doesn't really provide for “no, really, I just want the full texts for everything in the tree of rev X”, for instance | 12:15 |
| jelmer | mgz: hmm, yeah. I just used bzip2 for consistency (because get_parent_map uses it), but perhaps I should give it some more thought. | 12:15 |
| spiv | jelmer: which is what my half-done (but working as a prototype) brHPSS for stacked branches branch helps fix. | 12:15 |
| spiv | jelmer: or rather, that branch just adds a rather useful special case in a slightly hacky way | 12:16 |
| spiv | jelmer: but the general idea of using record streams even more is a good approach I think | 12:17 |
| spiv | (That particular special case also sort-of addresses the “look up all the inventory and CHK pages server-side and just send me what I need” rather than making the client traverse the revision->inventory->chk pages->texts links) | 12:18 |
| jelmer | spiv: yeah, it would definitely be nice to have the server handle those lookups | 12:19 |
| spiv | So a problem with just providing an HPSS iter_files_bytes for lightweight checkouts that for large trees (i.e. many files, deep directories) it still requires the client to get full inventory (so finding all the chk pages), which currently needs umpteen round trips. | 12:20 |
| spiv | Ideally you'd use something like the texts-for-rev hack that unlanded branch adds and use that | 12:20 |
| jelmer | spiv: your texts-for-rev branch was what got me into all of this :-) | 12:20 |
| spiv | Except we don't have great tools for consuming record streams apart from inserting them into an acutal repo | 12:21 |
| spiv | And for the lightweight checkout case we'd be happy enough to just build the inventory object in memory, and then unpack the text records directly to the filesystem as we receive them | 12:21 |
| spiv | But the infrastructure to do that doesn't really exist yet AFAIK, although it's in principle possible. | 12:22 |
| jelmer | spiv: the lightweight checkout case is what I've been looking at so far - having a streaming Repository.iter_files_byes (done, but using bz2) and a working Repository.get_inventory should be sufficient for that afaict | 12:22 |
| spiv | (You could in the interim write the stream out to a temporary repo and delete it after the tree build) | 12:23 |
| jelmer | (checkout uses Repository.iter_files_bytes) | 12:23 |
| spiv | What precisely is Repository.get_inventory going to send over the wire for CHK repos? | 12:23 |
| spiv | Perhaps just a full inventory in the HPSS inventory-delta format? | 12:23 |
| jelmer | That's a good question | 12:24 |
| spiv | Because sending just the 'inventory' record isn't going to do you much good :) | 12:24 |
| jelmer | right, the other alternative would be using .serializer.write_inventory_to_string | 12:24 |
| spiv | "Hi, have the name of the chk root page, kthxbye" | 12:24 |
| jelmer | but that's going to use the old XML serializer which we want to get away from | 12:24 |
| spiv | Ugh, no | 12:24 |
| spiv | That also reinforces the rich-root and subtree watershed | 12:25 |
| spiv | Whereas inventory-deltas are a touch nicer IIRC | 12:25 |
| spiv | There's also code to encode a full inventory as an inv delta (it's needed for some corner case I forget) | 12:26 |
| spiv | s/also/already/ | 12:26 |
| spiv | So I'd suggest reusing that. | 12:26 |
| jelmer | I'll have a look at that, thanks. | 12:26 |
| spiv | Also, it'll push you more in the direction of record streams :) | 12:26 |
| spiv | Which is really where we want to be, I think | 12:27 |
| spiv | They are intended to be an abstraction that works well both as an internal API and over the wire. | 12:27 |
| jelmer | yeah, they're quite reasonalbe | 12:28 |
| jelmer | the Repository.iter_revisions / Repository.iter_files_bytes HPSS calls are both very much oriented towards record streams (don't return in a specified order, allow for absent records) | 12:28 |
| spiv | So I'd try to keep in mind that there are separate concerns: | 12:28 |
| spiv | 1) How to deliver records over the wire: record streams. 2) How to efficiently ask the server which records to send for a given use case: case-by-case improvements needed :) | 12:29 |
| spiv | Incidentally, I reckon if you finish my branch so that 'bzr branch --stacked HPSS_URL' becomes fast, it will be a great thing to recommend for many uses people currently use lightweight checkouts for | 12:31 |
| spiv | (Not all, but certainly some) | 12:32 |
| spiv | I suspect there are folks that would rather have a real branch than a checkout, so long as they don't have to pay the full cost of all history (and having a bit more data cached locally, data that's actually relevant to making e.g. 'bzr diff' run fast, is also a plus) | 12:33 |
| spiv | Folks that just want a really cheap readonly mirror of something that they want to be able to update to the current tip periodically will of course be better off with a lightweight checkout | 12:34 |
| spiv | But I know I'd love to be able to do e.g. 'bzr branch --stacked lp:gcc' and have it get me a usable branch in time and bandwidth not much more than the size of the eventual tree on disk. | 12:35 |
| jelmer | spiv: I can't agree more | 12:36 |
| jelmer | vila: I'm getting build failures with sphinx on precise, have you seen those? | 12:39 |
| jelmer | s/build/test/ | 12:39 |
| jelmer | TypeError: expected string or Unicode object, NoneType found | 12:40 |
| jelmer | while trying to pickle something | 12:40 |
| === Quintasan_ is now known as Quintasan | ||
| mgz | hm, I don't get bzr-builddeb bug mail. | 13:36 |
| jelmer | mgz: you can subscribe :) | 13:39 |
| mgz | I'm used to mail magically appearing :) | 13:43 |
| fullermd | I certainly get mail all the time as if by supernatural agency. But I'm pretty sure it's more of the infernal than divine sort... | 13:43 |
| vila | jelmer: meh, those will be... interesting, do you have more info ? | 14:02 |
| vila | jelmer: not seen on babune apparently, may be the dependencies are not installed there | 14:03 |
| * vila tries to remember what bell is ringing... | 14:04 | |
| jelmer | vila: e.g. http://pastebin.ubuntu.com/745133/ | 14:04 |
| jelmer | vila: you were at the front door wondering why there was nobody there ? :) | 14:04 |
| vila | LOL | 14:05 |
| vila | I've never seen the bottom of this traceback but the top of it... well, sphinx internals have evolved so I'm not that surprised (a bit disappointed though :-/) | 14:06 |
| vila | sphinx or docutils | 14:06 |
| vila | hmm, and oneiric /usr/lib/pymodules/python2.7/sphinx/environment.py is different as line 250 doesn't match the one you're reporting for precise... | 14:09 |
| jelmer | I have 1.0.8+dfsg-2 | 14:10 |
| vila | I have... no sphinx in my schroot precise ;) | 14:10 |
| vila | jelmer: where do you get that ? On your local setup or in some build env ? | 14:12 |
| mgz | is the OOM from commiting something small in repo with big files repacking related? | 14:16 |
| jam | mgz: most likely | 14:19 |
| jam | "bzr commit" takes less memory than "bzr pack" | 14:19 |
| jam | though bzr-2.4 should do better in that regard | 14:19 |
| jam | (lower peak during pack) | 14:19 |
| jelmer | vila: this is on my local setup | 14:24 |
| jam | mgz: I meant to mention it, but then deleted the email before responding :) | 14:24 |
| jam | jelmer: my comment was a basic comment about your hpss series of changes, not specific to one patch | 14:25 |
| jam | I might have just missed the testing | 14:25 |
| vila | jelmer: ok, file a bug then, we strictly support sphinx only for the host that generates doc.bazaar.canonical.com, but the plan is still to switch to sphinx... when time permits | 14:25 |
| jam | I wasn't looking super close, but I thought a few of them didn't have any changes in a test_* file. | 14:25 |
| jelmer | jam: All of them should be updated at least bzrlib.tests.test_smart *and* bzrlib.tests.test_remote. I'll double-check | 14:25 |
| jelmer | *updating | 14:25 |
| GRiD | hi | 14:49 |
| === yofel_ is now known as yofel | ||
| mgz | hey GRiD | 14:56 |
| vila | jelmer: hmm, saying only doc.b.c.c is the only place where we care about sphinx may be over-optimistic, iirc, windows and osx installers build their own doc | 15:01 |
| vila | jelmer: filing a bug is even more the way to go | 15:01 |
| jelmer | vila: will do | 15:12 |
| jelmer | vila: looks like it's caused by lazy_regex | 15:28 |
| * vila pulls his remaining hairs | 15:28 | |
| mgz | ehehe | 15:30 |
| jelmer | I'm submitting a patch to make lazy regex compiled expressions pickleable | 15:30 |
| * fullermd makes a note to keep his head away from vila. | 15:30 | |
| * vila notes the backup plan | 15:30 | |
| mgz | jelmer: just thunk on __reduce__ maybe? | 15:31 |
| vila | jelmer: won't it be simpler to delay importing the real 're' and just get rid of lazy regexp in this context ? | 15:34 |
| mgz | it's worth not making lazy regexprs blow up pickle anyway | 15:35 |
| vila | hmm, may be, but I don't think we use pickle anywhere close regexps in bzr otherwise, whatever is simpler is fine by me | 15:36 |
| jelmer | mgz: re registers a custom type handler, so __reduce__ doesn't get called as far as I can tell | 15:40 |
| mgz | jelmer, just something like this isn't enough? | 15:44 |
| mgz | def __reduce__(self): return (re.compile, (self.pattern,),) | 15:44 |
| jelmer | mgz: hmm, that's a good point | 15:47 |
| jelmer | mgz: I was looking at unpickling back to a lazy compiled regex, but that's not really necessary | 15:48 |
| mgz | in practice, return (_real_re_compile, self._regex_args) ...but need to fold in kwargs first | 15:48 |
| * jelmer settles for __getstate__ / __setstate__ instead | 15:59 | |
| === beuno is now known as beuno-lunch | ||
| === luks_ is now known as luks | ||
| === luks is now known as luks_ | ||
| === luks_ is now known as luks | ||
| === beuno-lunch is now known as beuno | ||
| lifeless | jelmer: pickle, really? | 18:18 |
| mgz | need a break for food before finishing this off | 19:25 |
| trashbird1240 | hello | 19:30 |
| trashbird1240 | I accidentally pushed a branch to a repository root | 19:30 |
| trashbird1240 | can I undo it somehow? | 19:30 |
| trashbird1240 | (or have I not done anything harmful?) | 19:30 |
| jelmer | lifeless: I'm afraid so | 19:41 |
| jelmer | trashbird1240: you can remove the branch from that location by using "bzr rmbranch LOCATION" | 19:41 |
| jelmer | that should keep the repository intact | 19:41 |
| trashbird1240 | jelmer: okay, let me try it | 19:42 |
| jelmer | lifeless: (this was for the benefit of sphinx, not because I'm planning to use pickle myself) | 19:42 |
| trashbird1240 | wait: is LOCATION a directory? url? what is it? | 19:42 |
| jelmer | trashbird1240: local file path or url | 19:42 |
| trashbird1240 | okay, so my branch (the one that I accidentally pushed ) is called "dynamics" | 19:43 |
| trashbird1240 | do I go to the root of the repository and use "bzr rmbranch dynamics"? | 19:43 |
| trashbird1240 | let me check the help... | 19:43 |
| jelmer | trashbird1240: presumably if you ran "bzr push FOO" you would now run "bzr rmbranch FOO" | 19:43 |
| glyph | trashbird1240: branches aren't called things | 19:44 |
| glyph | trashbird1240: they have URLs :) | 19:44 |
| trashbird1240 | okay, bzr remove-branch gave me no output, so I presume it worked ;) | 19:44 |
| trashbird1240 | I pushed to a local directory | 19:44 |
| jelmer | trashbird1240: if you run it again, it should tell you there is no branch to remove | 19:44 |
| trashbird1240 | i.e. the repository is in /var/www/<place where my brancehs go> | 19:44 |
| trashbird1240 | OK... | 19:44 |
| trashbird1240 | bingo | 19:45 |
| trashbird1240 | I got "not a branch" | 19:45 |
| trashbird1240 | Thanks! | 19:45 |
| glyph | jelmer: so uh, if I do 'bzr branch . foo', is it going to treat the repository in the branch as a shared repository? | 19:46 |
| glyph | (assuming no shared repository exists above .) | 19:46 |
| jelmer | glyph: only if the repository in . is marked as shared | 19:47 |
| jelmer | (created with "bzr init-repo") | 19:48 |
| glyph | jelmer: interesting | 19:48 |
| glyph | jelmer: is there a series of commands that will make that happen, or do I have to mess with files in .bzr? | 19:48 |
| glyph | ('touch .bzr/repository/shared-storage' if I understand the magic correctly?) | 19:49 |
| jelmer | glyph: touch .bzr/repository/shared-storage | 19:49 |
| jelmer | though "bzr reconfigure" might also have an option that will do that for you | 19:49 |
| glyph | jelmer: no option I can see in the --help | 19:50 |
| lifeless | jelmer: sphinx *pickles* stuff? wtf? | 19:58 |
| jelmer | glyph: I can't either, so I've filed bug 893312 | 20:38 |
| ubot5 | Launchpad bug 893312 in Bazaar "reconfigure can not make a repository shared or unshared" [Medium,Confirmed] https://launchpad.net/bugs/893312 | 20:38 |
| jelmer | lifeless: the build() method in sphinx pickles the build environment and saves it to disk | 20:39 |
| lifeless | jelmer: aieeeeeeeeeeeeeeeeee I'm blind | 20:41 |
| salgado | hi. I thought there was an easy way of committing the result of a merge using the message of the last merged revision, but I can't seem to find it? | 20:41 |
| jelmer | lifeless: :) | 20:46 |
| jelmer | salgado: I'm not aware of one.. | 20:46 |
| jelmer | salgado: it should be pretty easy to write a hook that does that though | 20:46 |
| lamalex | lifeless, ping- are you around? | 20:48 |
| salgado | jelmer, oh, ok, thanks. I might give it a go at some point :) | 20:50 |
| lifeless | lamalex: hi | 21:05 |
| lamalex | lifeless, im trying to use testtools but im getting a goofy error i dont really understand when i run my tests with python -m testtools.run | 21:14 |
| lamalex | http://paste.ubuntu.com/745394/ | 21:14 |
| lifeless | your test isn't importable | 21:19 |
| lifeless | probably you have the name 'foo.py' | 21:19 |
| lifeless | 'foo.py' is not importable. | 21:19 |
| lifeless | 'foo' is | 21:21 |
| lifeless | ah - launchpad-tests.py | 21:21 |
| lifeless | rename that launcher_tests.py; testtools.run launchpad_tests | 21:21 |
| lamalex | launcher | 21:21 |
| lamalex | but really? | 21:21 |
| lamalex | it's a python file | 21:21 |
| lamalex | why wouldn't it end in .py | 21:21 |
| lamalex | like every other python file | 21:22 |
| lifeless | it should | 21:22 |
| salgado | /afk | 21:22 |
| === salgado is now known as salgado-afk | ||
| lamalex | is launchpad_tests a class name/ | 21:23 |
| lifeless | no, its a module name | 21:24 |
| lifeless | launcher_tests | 21:24 |
| lifeless | do this | 21:24 |
| lifeless | $ python | 21:24 |
| lifeless | >>> import launcher_tests | 21:24 |
| lifeless | it will fail | 21:24 |
| lamalex | ah ok | 21:26 |
| lamalex | i got it | 21:26 |
| lifeless | jelmer: so for clarity - sphinx upstream went insane, right ? | 22:31 |
| lifeless | jelmer: can we get it rolled back in precise? And send some stormtroopers to the authors? | 22:31 |
| jelmer | lifeless: they're not pickling the entire docs, just some bits of the configuration | 22:45 |
| jelmer | lifeless: it's not that bad, is it? | 22:46 |
| lifeless | jelmer: you tell me, how do they handle the config changing? | 22:46 |
| lifeless | 2-way diff? | 22:46 |
| lifeless | 3-way? | 22:46 |
| jelmer | lifeless: I have no idea | 22:46 |
| lifeless | It seems like a likely source of failure-to-update bugs | 22:47 |
| lifeless | as well as making the sphinx environment suspectible to arbitrary code execution, so becomes security sensitive | 22:47 |
| lifeless | so yeah, I think its pretty bad | 22:47 |
| lifeless | admittedly not as bad as the python json modules default-arbitrary-code-execution facility | 22:48 |
| jelmer | lifeless: hmm, /usr/lib/pymodules/python2.7/sphinx/environment.py:66 | 22:48 |
| jelmer | # This is increased every time an environment attribute is added | 22:49 |
| jelmer | # or changed to properly invalidate pickle files. | 22:49 |
| jelmer | ENV_VERSION = 39 | 22:49 |
| jelmer | lifeless: I don't think there are situations where we open random pickle files though | 22:49 |
| jelmer | s/don't think/know/ | 22:49 |
| jelmer | ehm, know there aren't | 22:49 |
| lifeless | if I can alter your pickle file, I can make you run arbitrary code. | 22:50 |
| lifeless | Its precisely as safe as 'eval <arbitrary file>' | 22:50 |
| jelmer | lifeless: sure | 22:50 |
| jelmer | lifeless: but this is just data generated by a local version of sphinx, stored for later reevaluation | 22:51 |
| lifeless | jelmer: I understand that; I don't understand why they need that, and it is insecure | 22:52 |
| jelmer | lifeless: whether it is insecure depends on how they use the pickle file, and in the overall setup, who can write to it | 22:52 |
| spiv | lifeless: configuring sphinx already involves arbitrary code | 22:52 |
| spiv | lifeless: so I'm not sure it's any less safe security-wise, but that certainly means its unsafe for robust pickling :) | 22:53 |
| lifeless | spiv: :) | 22:53 |
| lifeless | jelmer: its fundamentally not my problem, at least I know now to avoid sphinx for all my projects. | 22:53 |
| lifeless | jelmer: yes, I am willing to not use a tool because it uses pickle. | 22:54 |
| spiv | (or perhaps my hazy memory of how sphinx works is wrong, but I have a pretty strong gut feeling that pickles are a bad idea here) | 22:54 |
| mwhudson | spiv: -here | 22:54 |
| jelmer | lifeless: I don't like pickles either, but I don't see how it is necessarily a security issue. | 22:54 |
| spiv | mwhudson: hmm, I'm not *entirely* against pickles | 22:54 |
| mwhudson | spiv: i like them in burgers, i guess? | 22:55 |
| spiv | mwhudson: there's a very occasional quick hack where they can be tolerable ;) | 22:55 |
| spiv | Hah | 22:55 |
| * mwhudson is making changes that take a few minutes to test, please excuse ongoing silliness | 22:55 | |
| lifeless | I think pickle is fine when you can be absolutely sure noone else can ever under any circumstance compromise the pickle. | 22:55 |
| spiv | mwhudson: you can't be sillier than all out for 47 :P | 22:56 |
| lifeless | e.g. when its a tempfile() you immediately unlink :) | 22:56 |
| mwhudson | spiv: hah | 22:56 |
| lifeless | bbs | 22:56 |
| mwhudson | i think the idea that you can serialize arbitrary objects is a fairly hair-raising one | 22:56 |
| mwhudson | spiv: even ponting scored some runs last time though | 22:56 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!