[00:41] <stewart> hi all! Help! We appear to have a broken BZR repository, we've tracked it down to lp:percona-server/5.1 revision 459, which was merging in a bzr-git branch. now people can't checkout lp:percona-server-5.1
[01:57] <fullermd> stewart: You should mention the error to make it a little easier for people...
[01:57] <fullermd> AssertionError: ('not present: %r', StaticTuple('', '', 'TREE_ROOT'))
[01:57] <stewart> AssertionError: ('not present: %r', StaticTuple('', '', 'TREE_ROOT'))
[01:57] <fullermd> That's happening during making the working tree.  So the data all transferred OK, just something in the latest rev makes dirstate all cranky.
[01:58] <stewart> fullermd, snap. that's it :)
[01:58] <fullermd> I can hack around it by doing a 'rm -rf *; bzr revert', and seem to get a good tree.
[01:58] <stewart> huh
[01:58] <stewart> weird....
[01:58] <stewart> and concerning
[01:59] <fullermd> I don't know where to start on digging up what in the rev is doing unexpected things to dirstate.
[01:59] <stewart> at some point, this is going to completely blow up in our regression testing system.
[01:59] <fullermd> It DOES create a fair bit of the tree before blowing up, but I don't know why.
[02:00] <fullermd> Well, it only happens on creating the tree from scratch.
[02:00] <fullermd> I can `update -r` among revisions fine, and revert has no trouble.  But doing an initial creation of the WT (e.g., bzr remove-tree ; bzr co .) croaks.
[02:02] <fullermd> That's as far as I can go with it.  It may be enough to give a good start to someone who knows the code though.
[02:03] <fullermd> vila or jelmer may be around in 6 or 8 hours; they can probably help you more.
[02:03] <fullermd> But at least you can work around it.
[02:04] <stewart> fullermd, yeah, it'd be nice to work out how to fix this... I'm currently thinking of re-creating history since the problem revision and overwriting the tree.
[02:14] <wgrant> stewart, fullermd: An HTTP branch worked fine for me.
[02:14] <wgrant> bzr+ssh did not
[02:14] <stewart> wgrant, okay... that's weird.
[02:14] <wgrant> Rather
[02:15] <wgrant> Retrying to confirm...
[02:16] <spiv> wgrant: that is very weird, if true!
[02:16] <spiv> Definitely sounds like something wonky in tree building/TreeTransform
[02:16] <wgrant> Ah
[02:16] <wgrant> It was also bzr 2.1.4 on the host that worked
[02:17] <wgrant> That might be relevant
[02:17] <spiv> wgrant: ooh, a regression then?  That's a useful hint!
[02:17] <wgrant> I'll retry HTTP with something newer
[02:19] <fullermd> Yeah, I can't imagine how http vs ssh can make a difference, since it's happening well past when the data's transferred...
[02:20] <wgrant> We had a similarish problem a couple of months back
[02:21] <wgrant> Where something was corrupt and confusing the smartserver to not send enough
[02:21] <wgrant> IIRC
[02:26] <fullermd> I'd expect the updates and reverts to have trouble too if it were really short data.  Seems like just checkout that's croaking.
[02:26] <wgrant> Indeed
[05:07] <AfC> Does the [bzr] push-and-update plugin have a proper Debian package?
[05:37] <nigelb> lifeless: I got it fixed. Basically, I hadn't done bzr launchpad-login for my tarmac user :)
[08:14] <mgz> morning
[10:08] <LeoNerd> [offtopic]: does anyone know if there's something up with launchpad's bzr sync lately? I committed something to a personal repo on Friday, but LP still hasn't mirrored it yet.
[10:09] <LeoNerd> bzr revno lp:pangoterm => 480   vs   bzr revno http://bazaar.leonerd.org.uk/code/pangoterm => 482
[10:09] <mgz> LeoNerd: there was a window during the datacenter move where things were slightly borked
[10:09] <LeoNerd> Hrm.. I knew about the move so I didn't get too upset over the weekend, but ought it to have caught up by now?
[10:10] <mgz> what I'm not sure is if this particular thing is still borked, or if it will get updated shortly/next time remote changes
[10:10] <LeoNerd> OK,.. well I'll keep an eye on it a few days more
[10:18] <mgz> LeoNerd: getting fixed now.
[12:20] <jam> LeoNerd: looking here: https://code.launchpad.net/~leonerd/pangoterm/trunk
[12:20] <jam> it seems to think it 'successfully' tried to copy the branch on Saturday
[12:21] <LeoNerd> Yah :/
[12:21] <jam> it is already in the queue of things to be checked
[12:21] <LeoNerd> Hrmm.. OK
[12:22] <LeoNerd> Oh, it was 06:23 (guessing UTC?), so that might be before I committed it.. it may have been saturday morning when I actually ci'ed the code
[12:23] <jam> LeoNerd: most likely UTC, yes.
[12:26] <jam> LeoNerd: we currently have 9 jobs running branch copies, however, we can sometimes get into 3000 ish branches in the queue
[12:26] <jam> so I don't have a specific eta for your branch.
[12:27] <LeoNerd> Ahh OK. I'm in no rush, was just making sure it hadn't broken
[14:39] <mgz> jam: have we got any sane way of detecting if a rev/branch has an issue with giant binary files?
[14:40] <mgz> you did some manual discovery for the leo-editor problem
[14:41] <mgz> Lelak has a problem where currently trying to use his shared repo triggers OOM on repack, but there are 100 branches and picking out the non-borked ones to a fresh repo doesn't seem easy
[14:41] <jam> mgz: I think I looked at the index files using "bzr dump-btree --raw ...tix" and looking for things with large ranges
[14:43] <mgz> okay, what you did before: https://answers.launchpad.net/bzr/+question/175127#comment-4
[14:43] <mgz> current problem: https://answers.launchpad.net/bzr/+question/205362
[14:45] <Lelak> Thanks mgz!
[14:46] <mgz> the issue is we don't have any way of fixing this apart from manual intervention by someone who understands the repo format?
[14:48] <Lelak> mgz: this was for me?
[14:50] <mgz> jam: so, do we have any practical suggestions at all?
[14:51] <fullermd> It seems like it would be reasonably easy to bung up a plugin that does a foreach(rev in repo) { print size of change in bytes }.  Though it'd probably be awful slow...
[14:52] <jam> mgz: I don't have a simple script that detects it, but iterating over branch.repository.texts.keys() (and some associated info in the index there) would certainly be possible.
[14:56] <mgz> that sounds like a reasonable thing, what I'm not clear on is the details of a function that would do rev -> size of texts, compressed size of texts
[14:56] <jam> mgz: note that the (file_id, revision) tuple for a text is the same revision_id as the actual commit revision.
[14:57] <jam> so the mapping to revision is pretty trivial, IMO
[14:57] <jam> if you are looking at "what revision introduced X" it is X's revision_id :)
[14:57] <jam> if you are looking for "what data did rev X introduce" that is a bit harder, but we already have "get_revision_delta" and friends for that
[14:58] <mgz> hm, so I guess the real desired output is "what is the last rev of this branch that doesn't have giant texts"
[14:59] <jam> mgz: And I'm saying it by answering the "when were the giant texts introduced" and then just get_parent_map(introduced)
[14:59] <mgz> won't catch OOM issues from a 1MB jpg changing every rev, but should get the common case we care about
[14:59] <mgz> okay, I'll have a stab at that.
[15:16] <Lelak> mgz: after making the new branches to the new clean shared repo now I´m getting this error: bzr: ERROR: Parent not accessible given base
[15:17] <Lelak> How is the right procedure to avoid this error when branching from the bronken repo?
[15:18] <mgz> Lelak: I think you just want to reset the parent
[15:18] <Lelak> sorry I don´t know how to do this
[15:18] <Lelak> =(
[15:18] <Lelak> editing the branch info?
[15:18] <Lelak> text file inside the .bzr folder?
[15:19] <mgz> Lelak: can use `bzr config parent_location`
[15:20] <Lelak> thank you mgz!!! sorry for my ignorance
[15:20] <mgz> to get the location of in the old branch, and then use = to set it in the new branch
[15:21] <Lelak> I was looking your talk with jam. Do you think you would be able to have a script that tells where the problematic revision is?
[15:23] <mgz> Lelak: yup
[15:25] <Lelak> GREAT!!!
[15:25] <Lelak> :DDD
[17:42] <mgrandi> how do you deal with conflicts when
[17:44] <mgrandi> all the programs im using want the left and right file, but bzr wants the changes to be in the 'this' file?
[17:45] <mgrandi> aka i have 3 files but the programs want two, and modify those files instead of the one bzr looks in
[17:53] <mgz> mgrandi: see bzrlib/mergetools.py perhaps? or ask on the mailing list.
[17:55] <mgz> there are various different example tools that take some of this/this_temp/other/base/result as args depending on what they expect
[17:56] <mgz> jam: okay, so the core question I still don't know how to answer on the which-revs-are-bad front, but I threw together a plugin that does... something
[17:58] <mgrandi> yeah, i guess i was confused on what the 'result' file should be
[17:58] <mgz> probably not correct for the general case, but has a go: lp:~gz/+junk/bzr-repobloat
[17:58] <mgrandi> just the file without the .base .other .this extensions?
[17:58] <mgz> mgrandi: yup.
[17:58] <mgz> the one that actually contains the conflict markers
[17:59] <jam> mgz: as a small point, most plugins put __init__.py in the top level dir, so that you can just 'bzr branch' them into your plugin dir.
[18:00] <jam> mgz: and if you are going to lock the branch, you probably just want to do that at the beginning in cmd_*, unlocking and then re-locking might clear some of the caches.
[18:01] <jam> your sorting is also a bit odd, but otherwise that seems to do what you were wanting (taking the compressed size as the key to indicate how bloated it makes things)
[18:03] <mgz> the keys being _basis_end and _delta_end is confusing, but I'm assuming delta follows straight on?
[18:03] <mgz> I was a bit worried because one of the larger values on the testtools repo was just changing four copyright headers
[18:05] <mgz> making it produce useful output is just some more work to try harder to give useful info on the bad revs
[18:07] <mgz> should bump the lower bound to 16MB and make it a param. anything else past that? there's no easy way of getting the decompressed size I take it (not unpacking all texts is a goal)
[18:07] <mgz> ...and I need to leave
[18:07] <mgz> wgz is on to log though
[19:29] <danm_> Do either the pipeline plugin or the loom plugin allow you to re-order your changes?
[19:31] <danm_> Along the lines of "hg qpush --move some-patch" ?