[14:49] <LeoNerd> So.. Nobody has any thoughts about my broken shelf problem? Am I going to have to manually unpick the pack file myself?
[14:50] <mgz> I think I had a way back, but has been a while and I'm not sure I remember it I'm afraid
[14:50] <LeoNerd> It looks manually -possible-, just a little awkward. the changes that are shelved are one newly-added file (easy to pull out), and some inserts/deletes of lines in another file
[14:51] <LeoNerd> The inserts look easy enough. the deletes look the hardest bit to unpick manually, because I'll have to count line numbers of the 'c' lines
[14:51] <LeoNerd> I think the entire pack is only upset about the newly created directory, so if I can somehow split the pack file into two packs, one that creates the new file in the new directory and one that does the edits to existing files, that might make it simpler
[14:52] <LeoNerd> The new file is added in a new dir- and that's the one I can easily unpick by hand]
[14:53] <fullermd> Maybe you could try backing up to the rev where you'd shelved 'em, and unshelving there?
[14:53] <LeoNerd> That's not the issue.
[14:53] <LeoNerd> I don't think
[14:53] <LeoNerd> I think the issue is that I shelved adding a file to a new directory, but I didn't shelve adding that directory
[14:53] <fullermd> Seems like I have vague memories of some dance like that working once upon a time.
[14:55]  * LeoNerd will give it a go anyway
[14:56] <LeoNerd> Nope.. still upset about:  bzr: ERROR: No final name for trans_id 'new-3'
[14:59] <LeoNerd> Is the pack format actually documented somewhere? I feel it ought to be justabout possible to manually split this file into two pieces.. one that adds a file and one that has all the other-file edits
[14:59] <LeoNerd> I suspect it's only the former that's upset, but that one is easy for me to manually unpick
[14:59] <LeoNerd> I'm just not -quite- sure I understand how to parse the metadata and attribs sections at the top of the file
[15:00] <fullermd> Apart from the source (hopefully the comments in), I don't think so.
[15:00] <LeoNerd> http://paste.debian.net/883914/
[15:00] <LeoNerd> is the header
[15:00] <LeoNerd> I've sortof worked out those lines that begin d
[15:00] <LeoNerd> They appear to be a decimal number then a comma then that number of bytes... a list of strings. apparently an even-sized kv assoc list
[15:01] <fullermd> Yeah, it's some sort of KLV.
[15:02] <LeoNerd> The attribs one is harder. It seems to be 'd', a 10-byte string ("_id_number"), then the next stream character is 'i' which I'm not sure what it means
[15:03] <LeoNerd> maybe it's an "integer" value, ... which could be 30, ending at the 'e'?
[15:05] <fullermd> I think it's bencoding?
[15:05] <LeoNerd> https://en.wikipedia.org/wiki/Bencode ?
[15:06] <fullermd> Yeah.
[15:06] <LeoNerd> Ah.. that looks useful, thanks
[15:06] <LeoNerd> OK that looks quite simple. I wonder if I could write me up a quick bencode <-> JSON converter to make it easier to manually manipulate
[15:07]  * LeoNerd , unrelatedly, still sad that pastie.org seems to have died :(
[15:07] <fullermd> Well, considering bzrlib has a bencode implementation, and I'm sure somebody has written a JSON implementation for python sometime...
[15:07] <LeoNerd> By far the best pastebin I used
[15:09] <fullermd> Actually, in a little poking, bzrlib's implementation (at least the .py version; it uses a .pyx mostly) looks like just an import/edit of the code from some bittorrent implementation anyway.
[15:09] <LeoNerd> Mmm :)
[15:11] <fullermd> e.g., compare bzrlib/util/_bencode_py.py with http://www.math.uiuc.edu/~gfrancis/illimath/windows/aszgard_mini/pylibs/bzrlib/util/bencode.py
[15:11] <fullermd> Take out some of the class wrapping and the like...
[15:12] <LeoNerd> Eh; it looks quite easy and trivial enough to reïmplement
[15:12]  * LeoNerd was going to use it as an excuse to add another example into Parser::MGC anyway
[15:13] <fullermd> Pshaw.  Obviously you'd build it on Marpa...
[15:13] <LeoNerd> Hellno
[15:14] <fullermd> But then, there's apparently Convert::Bencode already too
[15:15] <LeoNerd> Yah probably. Again, it's an excuse to write example code
[15:15] <LeoNerd> Because people always complain I don't have enough
[15:15] <fullermd> That leads to recursion, though.  You start writing some sample code, you shelve part of it, and then suddenly...
[15:16] <LeoNerd> Then suddenly I'm stuck under a car fixing bits of it in order to change a lightbulb ;)
[15:17] <fullermd> In point of fact, I did exactly that a couple years ago    :p
[15:17] <fullermd> I had to go buy a switch to replace a stupid dimmer, and on the way back my brake caliper locked up.
[15:33] <LeoNerd> Hrmmmmm...
[15:33] <LeoNerd> d7:message13:State objects11:revision_id54:leonerd@leonerd.org.uk-20161018213449-8qaa6cpx1l9ph0j9eB1063   <== is the metadata field
[15:33] <LeoNerd> It's a dict of  message => "State objects", revision_id => (that long string)  which ends at the 'e' and then that  B1063  seems to be trailing
[15:34] <LeoNerd> Any idea what that is?
[15:34] <LeoNerd> If I stop at that e then it's fine
[15:36] <LeoNerd> Ooh... maybe I wasn't supposed to include that.. because before that, there was a  B98  marker
[15:38]  * LeoNerd mumbles that pack format doesn't linefeed-terminate its blocks
[15:38] <fullermd> B looks like a marker in the pack header.  At least in a quick glance at the code, which I can't read.
[15:39] <LeoNerd> Yeah.. the pack file itself seems to be a one-line header, followed by data blocks that are B then a decimal integer giving the byte count then a linefeed then that many bytes
[15:39] <LeoNerd> but no linefeed after the data block payload, meaning that the next B marker for every subsequent block starts at the end of the previousl ine
[15:41] <LeoNerd> http://paste.debian.net/883932/  <== is my decoded attribs section anyway... this appears to be working :)
[15:42] <LeoNerd> So.. I wonder if we can work out from here what the error message means
[15:43] <fullermd> Oh, shucks, you can do that by deduction from the message itself   :p
[15:44] <LeoNerd> I might if I knew what a "final name" is
[15:44] <fullermd> Or rather, you can assume something plausible, and since there's nobody around to contradict it, you're right by default.
[15:44] <LeoNerd> Oh.. that little "_new_name" chunk?
[15:45] <LeoNerd> Ah.. maybe I just have to add  'new-3' => 'Room'  into that
[15:46] <LeoNerd> Oh.. and _new_parent otherwise it doesn't know where to root it
[15:46] <fullermd> The problem, as best I guess, is that new-%d is a standin file id shelf puts on things that don't already have one.
[15:46] <fullermd> So since the dir wasn't committed yet when the file was shelved, it didn't have one, so got a standin.  Now there's nothing in the tree with that standin, so it has no idea where to put the file.
[15:47] <LeoNerd> Hmmm
[15:47] <fullermd> So, in _theory_, if you horked up the transform in the pack so it tried to put it under the actual file-id of the directory...
[15:47] <LeoNerd> But it has a new id
[15:47] <LeoNerd>                          'new-3' => 'room-20161015171947-i9zc40ts93rt5svr-1',
[15:48] <fullermd> Is that the directory?
[15:48] <LeoNerd> lib/Net/Async/Matrix/Room                          room-20161015171947-i9zc40ts93rt5svr-1
[15:48] <LeoNerd> According to   bzr inventory --show-ids
[15:48] <LeoNerd> and the dir exists
[15:48] <fullermd> See?  If you'd just kept quiet, I would have been right.
[15:49] <LeoNerd> Hah
[15:50] <LeoNerd> I think I may start by splitting the shelf in half anyway, see if I can make those two separate pieces to simplify things
[15:57] <fullermd> Having all those smarts in shelves is really nice, until it isn't   :|
[15:59] <LeoNerd> Ooooh
[16:00] <LeoNerd> So having nuked all mention of new-2 or new-3 and fixed up the byte lengths, unshelve --preview is happy
[16:00] <LeoNerd> and bzr unshelve  itself did it :)
[16:00] <LeoNerd> Woo.. this seems fine
[16:00] <LeoNerd> Now all that's missing is the entire content of the new file but I can cut that verbatim from the shelf file
[16:01] <LeoNerd> aaaaaand we're back
[16:02] <LeoNerd> ofcourse my extracted code doesn't even compile let alone pass its tests, but that's my fault now. ;)
[16:02] <fullermd> Man, that whole procedure sounds like something you'd find in chapter 7 of "Git For People Smarter Than You".
[16:03] <LeoNerd> Anyhow thanks much for that bencode hint - that seemed to be the key to the whole process
[16:03] <LeoNerd> fullermd++
[16:04] <fullermd> Yay, I did something good.  I can go accomplish nothing the rest of the day then.
[16:04] <LeoNerd> :)
[21:14] <LeoNerd> meh.. now I'm too nervous to use the shelf with a pending mkdir hanging around
[21:38] <LeoNerd> Hrm... also I find myself wanting an  unshelve  with yes/no questions per chunk