| * jelmer waves | 00:16 | |
| mwhudson | jelmer: hi | 00:24 |
|---|---|---|
| mwhudson | jelmer: will the new bzr-hg improve memory behaviour at all for imports? | 00:25 |
| jelmer | mwhudson: somewhat - fetching in batches will limit the memory usage somewhat | 00:25 |
| mwhudson | jelmer: ok, so it'll be worth retrying the python import, for example? | 00:26 |
| jelmer | mwhudson, but we're still keeping the entire bundle we receive from the mercurial server in memory at the moment | 00:26 |
| mwhudson | ah | 00:26 |
| jelmer | mwhudson, are there any particular ones for which memory usage is an issue at the moment? | 00:27 |
| mwhudson | jelmer: dunno, i just saw a bug report update go by | 00:27 |
| mwhudson | https://answers.launchpad.net/launchpad/+question/160305 | 00:27 |
| mwhudson | jelmer: you'll be surprised to learn that the OO.org import is reported as having issues | 00:28 |
| jelmer | ah, openoffice | 00:28 |
| mwhudson | but also python | 00:28 |
| mwhudson | which is a bit less ridiculous i guess | 00:29 |
| jelmer | I hope it'll be better with the rollout | 00:30 |
| mgz | jelmer, for bug 788219 is what apport has duped it to what you were thinking of? | 00:53 |
| ubot5 | Launchpad bug 788219 in bzr (Ubuntu) "bzr crashed with IOError in get_password() (dup-of: 534326)" [Undecided,New] https://launchpad.net/bugs/788219 | 00:53 |
| ubot5 | Launchpad bug 534326 in Bazaar GTK+ Frontends "bzr: ERROR: gnomekeyring.IOError:" [Medium,Fix released] https://launchpad.net/bugs/534326 | 00:53 |
| jelmer | mgz, no, those are definitely different issues | 00:54 |
| mgz | because that's still a bzr-gtk bug and the guy doesn't have bzr-gtk installed | 00:54 |
| jelmer | they're both caused by gnomekeyring, but one is when it's used through launchpadlib and one when it's used through bzr-gtk | 00:54 |
| jelmer | they both have to handle IOError | 00:55 |
| jelmer | mgz, unduped - thanks! | 00:55 |
| mgz | can you find the launchpadlib bug? I didn't find it when I tried. | 00:55 |
| mgz | I don't know what to do about bug 791839 | 01:04 |
| ubot5 | Launchpad bug 791839 in bzr (Ubuntu) "bzr: ERROR: exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte ..ordinal not in range " [Undecided,New] https://launchpad.net/bugs/791839 | 01:04 |
| mgz | as reported it's a dupe. | 01:04 |
| poolie | jelmer: i think i've hit that too | 01:04 |
| poolie | the ioerror | 01:05 |
| mgz | I don't see what's wrong with Fabien's statement that etckeeper should use git. | 01:05 |
| poolie | i'm also experiencing lplib corrupting the credentials it stores in the keyring, which is a separate unrelated bug | 01:05 |
| mgz | bzr plain doesn't work on nix without LANG set and obeyed. | 01:05 |
| poolie | probably it would be better to assume the filesystem is utf-8 if it's not set | 01:06 |
| poolie | well, there are a couple of cases here | 01:07 |
| poolie | - the filename is just not in any valid encoding, or it's inconsistent with other files in the same tree | 01:07 |
| mgz | he's got random junk cert names. | 01:07 |
| poolie | which is an inherently hard case for us | 01:07 |
| mgz | even without the traceback from bzr we won't commit the file. | 01:07 |
| poolie | the other | 01:07 |
| poolie | is that the encoding is not set or not set properly | 01:07 |
| poolie | which is pretty much what you say in https://bugs.launchpad.net/bzr/+bug/715547/comments/7 | 01:08 |
| ubot5 | Ubuntu bug 715547 in Bazaar ""bzr add" crashed ERROR: exceptions.UnicodeDecodeError" [Medium,Confirmed] | 01:08 |
| poolie | mgz: are you sure they're not utf-8? | 01:09 |
| poolie | if they came from an ubuntu package and they're not utf8, that sounds a bit like a bug in that package | 01:09 |
| mgz | they could be, but there's no reason to expect all programs writing to /etc to behave | 01:09 |
| mgz | and having your backup randomly fall over in that case is... a hard behaviour to argue for | 01:10 |
| poolie | so | 01:10 |
| mgz | wereas falling over fast is generally the right thing for a manually curated source tree | 01:10 |
| poolie | i think we should group/dupe the bugs into those two splits | 01:10 |
| poolie | plus any others for accidental cases where we know the right encoding but just do the wrong thing | 01:11 |
| poolie | what do you think? | 01:11 |
| mgz | seems reasonable. we've had various symptom bugs for these problems in the past than have been fixed, regressed, and so on | 01:12 |
| poolie | i think "assume utf-8" should be feasible to do and by my recollection of the bugs reported it will fix somewhere well over half of them | 01:12 |
| poolie | like it seems like it will fix this case | 01:13 |
| poolie | "i want to version names with just random byte strings" might be harder | 01:13 |
| poolie | hm | 01:13 |
| poolie | if people really want that they could set their fs encoding to say 8859-1, which will accept anything and send it to equally arbitrary utf-8 | 01:14 |
| mgz | that's true. | 01:14 |
| poolie | i wonder how hard it is to persuade python to use an fsenc other than what it originally detected? | 01:15 |
| poolie | do you want to reorganize them or shall i? | 01:19 |
| mgz | changing it would be somewhat dodgy, comes from a global variable set during initialise that lots of things depend on | 01:23 |
| poolie | we// | 01:23 |
| mgz | ^go ahead, I think we shouldn't have too many open bugs for the same thing, maybe just want a new one for changing the default | 01:23 |
| poolie | i think at least having a bug saying "we want this behaviour" | 01:23 |
| mgz | yup. | 01:24 |
| poolie | then we can see if it's too hacky to actually get it | 01:24 |
| poolie | as a matter of (fairly) objective fact, the right fs name encoding is utf-8 | 01:24 |
| mgz | our uses probably, or at least should, all go through osutils anyway? | 01:24 |
| poolie | either do, or can be made too | 01:26 |
| poolie | s//to | 01:26 |
| poolie | possibly we can get around it by always passing byte strings | 01:26 |
| poolie | in the very worst case we can reimplement some os interfaces | 01:27 |
| === r0bby is now known as robbyoconnor | ||
| poolie | o/ spiv | 04:22 |
| spiv | Hey poolie | 05:44 |
| poolie | hi spiv | 06:35 |
| === hunger_ is now known as hunger | ||
| poolie | spiv, still here? | 07:36 |
| poolie | i'm trying to fix the lp-propose lazr.restfulclient bug | 07:36 |
| poolie | for some reason python insists on loading the OS version of that library even though i have another one on my pythonpath... | 07:37 |
| poolie | oh, maybe because it imports lazr.somethingelse | 07:40 |
| spiv | Maybe the lazr.* packages have funky .pth files or something? | 07:41 |
| poolie | ! | 07:42 |
| poolie | :qa | 07:42 |
| poolie | heh | 07:43 |
| poolie | /usr/lib/python2.6/dist-packages/lazr.restfulclient-0.9.20-nspkg.pth | 07:43 |
| poolie | blah | 07:43 |
| poolie | import sys,types,os; p = os.path.join(sys._getframe(1).f_locals['sitedir'], *('lazr',)); ie = os.path.exists(os. path.join(p,'__init__.py')); m = not ie and sys.modules.setdefault('lazr',types.ModuleType('lazr')); mp = (m or []) and m.__dict__.setdefault('__path__',[]); (p not in mp) and mp.append(p) | 07:43 |
| poolie | all on one line | 07:43 |
| fullermd | Hrm. Should bzr really check URL's before filenames? | 07:44 |
| poolie | fullermd: well, URLs are a strict superset of filenames | 07:44 |
| fullermd | Not when they fail :p | 07:45 |
| poolie | by which i mean, any filename can be represented as aurl but not vice versa | 07:45 |
| poolie | but, we can use sensible heuristics if it's bad | 07:45 |
| fullermd | % bzr diff 2001:470:8:7f6 | 07:45 |
| fullermd | bzr: ERROR: Unsupported protocol for url "2001:470:8:7f6" | 07:45 |
| poolie | right | 07:45 |
| poolie | ./2001... should fix it | 07:45 |
| jam | morning all | 07:46 |
| fullermd | Yeah, it's trivial to workaround. But annoying. | 07:46 |
| vila | hey guys ! | 07:46 |
| fullermd | Checking the other way around... well, I guess it would make your life really unpleasant if you had a file called 'lp:bzr' in your tree, but... | 07:46 |
| poolie | fullermd: i guess we could say "if there's no such scheme, assume it's a file" | 07:47 |
| poolie | that could work | 07:47 |
| fullermd | File a bug? | 07:47 |
| poolie | spiv: yes i think that's it: so just having this module installed looks ilke it will always be loaded? | 07:47 |
| mgz | fullermd wants to break bazaar on alternative data streams in a new and interesting way? | 07:50 |
| mgz | nice systems don't let people put colons in filenames :) | 07:50 |
| fullermd | Well, I've for sure never advocated breaking things in old and boring ways. I've got style, dangit. | 07:50 |
| poolie | :) | 07:50 |
| fullermd | I store important data in files called '%s', just to be sure they're not used in format strings. | 07:51 |
| mgz | ehehe | 07:52 |
| poolie | wow that's a precious snowflake | 07:54 |
| poolie | i did not realize you could run arbitrary code from there | 07:54 |
| fullermd | There are more things in heaven and on earth than are hacked apart in your philosophy ;) | 07:55 |
| vila | poolie: Regarding lp:~mbp/bzr/220464-stale-locks , is there some pending points you want to address or can it be landed ? | 07:55 |
| poolie | > Lines starting with import (followed by space or tab) are executed. | 07:56 |
| poolie | vila, hi | 07:56 |
| poolie | i haven't fixed the pqm failure yet | 07:56 |
| vila | poolie: ISTM this is already landable and the proposal is already > 900 lines | 07:56 |
| vila | ha, ok | 07:56 |
| mgz | poolie, I've got one more change in my copy of the branch, I didn't remove the win32 test skip | 07:56 |
| mgz | what was the pqm failure? it didn't get pasted to the mp | 07:57 |
| spiv | poolie: yes, it's a great way to do cool/surprising things :/ | 07:57 |
| mgz | pth files are evil. | 07:57 |
| vila | poolie: also, AIUI, this is now restricted to the same user so you may want to update the news entry | 07:58 |
| poolie | good point | 07:58 |
| mgz | okay, pushed, lp:~gz/bzr/220464-stale-locks | 07:59 |
| spiv | Presumably there's a convention for co-operating nicely with namespace packages like that for when you're developing your own version of a module in that namespace, but I'm not sure what it is. | 07:59 |
| poolie | hthanks | 07:59 |
| vila | poolie: python2.6 ? Aren't you running natty ? | 08:00 |
| poolie | i am, but that was from a clean maverick chroot | 08:00 |
| poolie | to see if it's something strange on my main system | 08:00 |
| poolie | the same problem occurs there | 08:00 |
| vila | ha ok | 08:01 |
| poolie | ok i'll leave this and finish 220464 | 08:04 |
| jam | poolie: where do you think you're at with the stale lock thing | 08:09 |
| jam | I feel like we've discussed it to exhaustion, without actually reaching consensus | 08:09 |
| vila | jam: really ? Did you read the last version on the proposal on lp ? | 08:11 |
| jam | vila: did you read the discussions between robert, martin and myself? | 08:11 |
| vila | yes | 08:12 |
| mgz | apart from esoteric things about people using funny network share setups with computers all named the same thing the issues look resolved to me | 08:12 |
| vila | if there are still controversial points on these cases then I think tests are better than discussion to break the ties | 08:12 |
| jam | mgz: except it doesn't seem all that esoteric ime. Specifically if you install ubuntu from live install, it always calls the machine "ubuntu" | 08:13 |
| mgz | but would you really mount a filesystem on another such system remotely? | 08:13 |
| jam | mgz: sftp:// | 08:14 |
| vila | sftp://ubuntu ? | 08:14 |
| jam | vila: sftp://bazaar.launchpad.net/my/project/branch | 08:14 |
| jam | poolie's proposal breaks *remote* locks that look like they were held by the local machine | 08:14 |
| jam | which has some nice bits | 08:14 |
| jam | like "bzr push sftp://... ^C bzr push" will work | 08:15 |
| jam | rather than warning you about the stale lock from the previous push | 08:15 |
| vila | right, my point is that writing tests for that will make the actual and intended behaviors clearer | 08:16 |
| jam | mgz: it has some problematic bits that if your machine is configured oddly, you can overwrite critical data. | 08:17 |
| jam | I'm trying to weigh the balance in my head | 08:17 |
| jam | since as poolie points out, breaking locks by hand can be just as risky | 08:17 |
| vila | and since the proposal is already 945 lines long it should either be landed or split or all discussions become more complex and progress harder | 08:17 |
| vila | AIUI the only case where we can override/lose data is for the repo lock | 08:18 |
| vila | in this case, the .pack ..ix files should still exist and we can think about ways to recover them but this is out of scope for this proposal | 08:19 |
| vila | also, this lock is short-lived to just pausing a bit may be enough to detect a stale lock, but I'd rather look at this kind of change in a smaller proposal | 08:20 |
| vila | s/to just/so just/ | 08:20 |
| jam | vila: we lose data from .conf overrwrites and a branch lock | 08:20 |
| jam | we lose master bound branch integrity from branch locks | 08:20 |
| jam | we lose a few other things. | 08:20 |
| jam | nothing I would say is "super critical", but it *is* data loss | 08:20 |
| jam | (my bound branch commit can succeed, and in the end master doesn't point at my revision.) | 08:21 |
| jam | vila: so the *critical* case is repo locking, but there are minor cases elsewhere | 08:21 |
| vila | then back to tests, if these scenarios are worrying enough, they should be covered no ? | 08:22 |
| jam | I'm pretty sure I'm falling on, "this is a net gain so we should land it", but I don't feel it is cut-and-dry | 08:22 |
| jam | vila: we can add tests for concurrency issues. However, we have to decide if we care if they fail | 08:22 |
| jam | vila: the current proposal will fail under esoteric conditions | 08:22 |
| jam | is that ok? | 08:22 |
| jam | I certainly can write a test that proves it. | 08:23 |
| jam | But does that actually help us. | 08:23 |
| vila | Yes it does, it means a better ground for addressing it | 08:23 |
| vila | It kills the FUD :) | 08:23 |
| jam | vila: except the discussion is tending towards "that failure mode is ok" | 08:23 |
| jam | which means, we don't actually need the test | 08:24 |
| jam | and actually, we don't *want* the test, because it would fail | 08:24 |
| jam | vila: we know from the code and its behavior how it can fail. we are asking "is a this rare enough event that we don't need to worry about it" | 08:24 |
| jam | no FUD | 08:24 |
| jam | just an open question | 08:24 |
| jam | we don't have to always be perfect, and we have to determine the tradeoffs | 08:25 |
| vila | I'm not saying you're propagating FUD, I'm saying you're a FUD victim here. To rescue you we should address your fear :) | 08:26 |
| vila | or doubts | 08:26 |
| jam | vila: I can't test how often in-the-wild the esoteric conditions are met | 08:27 |
| jam | which is the fear | 08:27 |
| jam | I don't care about the test | 08:27 |
| jam | I need to know real-world-how-likely-is-this-to-fail | 08:27 |
| jam | I'm thinking it is low enough to go ahead. | 08:27 |
| jam | I *know* it fails in a particular way, I *don't* know how likely that way is. | 08:27 |
| jam | We've had other cases where we made the tradeoff | 08:28 |
| jam | and people got bit by it. | 08:28 |
| jam | Like the "inventories during repack" | 08:28 |
| jam | I knew it was broken, but that "bzr pack" fixed it. | 08:28 |
| jam | People have hit it, and we decided that wasn't enough, so we spent more time to work on it. | 08:28 |
| vila | But isn't adding the user in the check enough to reduce the esoterism here ? | 08:29 |
| jam | vila: maybe, but the most likely cases for actually running into this is probably stuff like cron scripts | 08:29 |
| jam | I'm unlikely to race with myself | 08:30 |
| jam | we ran into a lot of bugs from the twisted guys, where they had multiple bots pulling into the same repository. | 08:30 |
| poolie | hi jam | 08:31 |
| jam | hi poolie | 08:31 |
| jam | poolie: so I think what I'm leaning towards is: land it, but file a couple bugs about known failure cases. | 08:34 |
| jam | To give us points to track if it happens in the wild. | 08:34 |
| jam | And especially a way to document known workarounds, etc. | 08:34 |
| poolie | i think so too | 08:38 |
| poolie | mm, except i'm against filing vague bugs | 08:38 |
| poolie | but if we say "will hit trouble if you have two machines with the same hostname" | 08:39 |
| poolie | you guys ready for a call in a `5m? | 08:44 |
| poolie | 15* | 08:44 |
| lifeless | jamesh: poolie: FWIW, I would lean towards being conservative; our user base is large enough now that 'minor risk' tends towards 'will happen', I think. | 08:48 |
| lifeless | s/jamesh/jam/ | 08:48 |
| poolie | mm | 08:48 |
| lifeless | but, I say this from the peanut gallery; the call is yours. | 08:48 |
| poolie | perhaps we should downgrade it to just warning, and leave it up to the user to decide if they think it's probabyl dead? | 08:49 |
| poolie | this may be a cop out | 08:49 |
| poolie | if they get it wrong anyhow | 08:49 |
| lifeless | thats what vim does, and it has a somewhat simpler problem to solve I tihnk | 08:50 |
| poolie | why simpler? | 08:50 |
| lifeless | no smart server | 08:52 |
| lifeless | just filesystem | 08:52 |
| lifeless | perhaps its equivalent | 08:53 |
| lifeless | I wasn't claiming radically simpler problem | 08:53 |
| jam | lifeless: well, their lock is also a WAL, which allows them to "incorporate changes" which we don't provide | 08:53 |
| vila | jelmer: time to reboot :) | 08:56 |
| poolie | WAL? | 08:58 |
| poolie | write ahead log? | 08:58 |
| jam | right | 09:00 |
| jam | I don't know that it is strictly that | 09:00 |
| jam | but you can say "apply changes from the .swp file" | 09:00 |
| jam | before you delete it | 09:00 |
| poolie | really, "apply", not just loading the whole thing? | 09:02 |
| jam | poolie: I don't know the specifics. But I think the idea is that if the text file is really big, Vim doesn't write out the changes to the file all the time. But keeps a small "these are the edits" in the .swp file. I just know that one of the options on load is "delete" and another is "recover" | 09:06 |
| poolie | spiv, hi? | 09:08 |
| poolie | i think 'recover' is just 'load it' not 'merge the changes' | 09:08 |
| poolie | imbw | 09:08 |
| jelmer | Riddell: bug 746822 | 09:53 |
| ubot5 | Launchpad bug 746822 in Launchpad itself "fails to build recipe with "bzr: out of memory"" [High,Triaged] https://launchpad.net/bugs/746822 | 09:53 |
| apw | anyone know what this might mean, when trying to bzr merge-package: | 09:58 |
| apw | NoSuchRevision: CHKInventoryRepository('file:///home/apw/local/bzr/tmpFFe1XH/upstream/.bzr/repository/') has no revision ('james.westby@ubuntu.com-20100820023033-dousm0xjfuraxtd7',) | 09:58 |
| jelmer | apw: merge-package apparently can't find a particular revision but it has a reference to it | 10:01 |
| jelmer | apw: it sounds like it might be a bug in bzr-builddeb or a broken repository | 10:03 |
| apw | jelmer, i can bzr checkout the url ok, just not merge it | 10:03 |
| jelmer | apw: does regular merge work? | 10:03 |
| jelmer | Riddell: bug 746822 seems to be the same issue as the recipe build running out of memory you mentioned | 10:04 |
| ubot5 | Launchpad bug 746822 in Launchpad itself "fails to build recipe with "bzr: out of memory"" [High,Triaged] https://launchpad.net/bugs/746822 | 10:04 |
| Riddell | jelmer: yep, not much extra information on it though | 10:04 |
| apw | jelmer, yet it would be mergable directly | 10:04 |
| jelmer | vila: does 2.3.3 really address just a single bug (failUnless & co cause PendingDeprecationWarning) ? | 10:10 |
| poolie | if so, that does not seem worth SRUing | 10:11 |
| poolie | it's true natty has 2.7 and people might run the tests there, but, it's not crucial | 10:12 |
| jelmer | fwiw: 2.3.2 hasn't been SRU yet either and has more bugs (12) | 10:12 |
| jelmer | though I'm surprised that we did a release just for the deprecation warning thing | 10:12 |
| poolie | ah, that makes more sense | 10:12 |
| poolie | i think it just squeaked in between 2.3.2 being tagged, and being discarded | 10:13 |
| poolie | it was cancelled during the release process for some reason that i forget at the moment | 10:13 |
| jelmer | ah, 2.3.2 was *that* release | 10:13 |
| jelmer | poolie: thanks | 10:13 |
| poolie | wasn't it? | 10:13 |
| poolie | ah the release page says | 10:13 |
| poolie | > This release broke the test suite under python-2.7 and the corresponding tarball has never been released. | 10:14 |
| vila | yup | 10:14 |
| poolie | so my fix makes it releasable | 10:14 |
| vila | yup and allowed the test suite to run which is part of the SRU AIUI | 10:14 |
| vila | well, at least for recent series | 10:14 |
| poolie | mm | 10:15 |
| poolie | but would it run against 2.7 on natty? | 10:15 |
| poolie | yes, it would, since 2.7 is the default | 10:15 |
| poolie | well that's all good then | 10:15 |
| vila | right, 2.3.2 was worth a SRU but couldn't released so I cut 2.3.3 since fixing the tag issue was not practical | 10:16 |
| vila | s/released/be released/ | 10:16 |
| poolie | ok | 10:23 |
| poolie | so the figurehead should be something from 2.3.2 probably | 10:24 |
| maxb | 2.3.2 nominally doesn't exist, right? | 10:24 |
| poolie | right, see above | 10:25 |
| poolie | and good morning | 10:25 |
| jelmer | poolie: what was the ecryptfs bug? | 11:13 |
| poolie | https://bugs.launchpad.net/ubuntu/+source/linux/+bug/793367 | 11:26 |
| ubot5 | Ubuntu bug 793367 in linux (Ubuntu) "ecryptfs corruption during make -j" [Undecided,Fix committed] | 11:26 |
| poolie | i sent a mail | 11:30 |
| poolie | ok that's enough for now | 11:30 |
| jelmer | poolie: thanks | 11:31 |
| fritz[] | hi | 11:37 |
| fritz[] | I'm trying to help with this bug | 11:37 |
| fritz[] | https://bugs.launchpad.net/bzr/+bug/680529 | 11:37 |
| ubot5 | Ubuntu bug 680529 in Bazaar ""Lock was renamed into place, but now is missing"" [High,Confirmed] | 11:37 |
| fritz[] | can someone instruct me where to put sleep? | 11:37 |
| jelmer | hi fritz[] | 11:42 |
| fritz[] | hi | 11:42 |
| poolie | jelmer: that's great, thanks | 11:43 |
| poolie | fritz[]: basically in lockdir.py, just look for the 'lock was renamed into place' mesasge | 11:46 |
| poolie | put that in a 'for i in range(5):', try the check, and if it fails, sleep(3) | 11:46 |
| fritz[] | http://bazaar.launchpad.net/~bzr-pqm/bzr/bzr.dev/view/head:/bzrlib/lockdir.py | 11:47 |
| fritz[] | from line | 11:47 |
| fritz[] | 251 | 11:47 |
| fritz[] | to | 11:47 |
| fritz[] | 262 | 11:47 |
| fritz[] | put that in for loop? | 11:47 |
| poolie | i would say only up to line 255 | 11:48 |
| poolie | then, if it doesn't find it | 11:48 |
| poolie | obviously you don't want to raise straight away, but only if the data is not found on any attempt | 11:49 |
| poolie | thanks very much for taclking this! | 11:49 |
| poolie | it seems to bite many people but it's hard to reproduce- | 11:49 |
| fritz[] | sure, no problem | 11:49 |
| fritz[] | I'll just take maseratti :) | 11:49 |
| jam | jelmer: so with bzr.dev "time bzr branch --stacked -Dmemory ...kde-workspace" only peaks at 150MB | 11:54 |
| jam | and takes 2.4min here | 11:54 |
| jam | that is http:// explicitly | 11:55 |
| jelmer | jam: which bzr version is that? | 11:55 |
| jam | bzr.dev | 11:56 |
| jam | I'm checking an older version next | 11:56 |
| jam | do you know what is on the builders? | 11:56 |
| jelmer | I think the builders are running 2.2-something | 11:56 |
| jelmer | let me check | 11:56 |
| jam | jelmer: using bzr-2.2.2, I get 2m32s, and 188MB peak memory for the same kde-workspace | 12:05 |
| jam | not all that different | 12:06 |
| jam | note, those numbers might ~double if they are using 64-bit systems | 12:07 |
| jam | though why you would run a 64-bit system with only 1GB of ram seems a bit silly. | 12:07 |
| jelmer | jam: virtual process size limit is set to 1 000 000 000 bytes | 12:07 |
| jam | given that 64-bit programs are almost universally much bigger in mem because of the extra pointer sizes | 12:08 |
| jelmer | I'm not sure why it uses RLIMIT_AS rather than something else | 12:08 |
| === mrevell is now known as mrevell-lunch | ||
| jam | spiv: with bzrr-2.2.2, 'bzr branch --stacked lp:.../kde-workspace' took 220MB peak, and 5m4s. While http:// took 188MB and 2m30s. | 13:41 |
| jam | so bzr+ssh vfs seems a bit weak | 13:41 |
| spiv | jam: yes, probably because there's double-caching | 13:46 |
| spiv | jam: e.g. RemoteRepository will keep a get_parent_map cache, then when it falls back to VFS the "real" repository object will probably fetch and cache most of the same info | 13:47 |
| spiv | (But in a different form, of course) | 13:47 |
| spiv | I think the primary fix there is "don't use VFS" | 13:47 |
| jam | spiv: branch --stacked, so I think most of the 'build-the-working-tree' stuff doesn't use get_parent_map | 13:47 |
| spiv | I'd be a little interested in if lp:~spiv/bzr/faster-stacked-tree-build goes better (needs to be run on both server and client) | 13:48 |
| spiv | (It doesn't totally eradicate VFS calls, but it does reduce them) | 13:49 |
| jam | spiv: hard to run that on LP :) | 13:49 |
| spiv | jam: local testing is faster anyway, right? ;) | 13:50 |
| jam | spiv: sure, but doesn't compare. Anyway, I do have a -Dhttp -Dhpss log of both, I might grep through it | 13:52 |
| jam | in theory, the same calls should be made | 13:52 |
| jam | spiv: it is also possible that my machine was more heavily loaded during one of the runs | 13:55 |
| jam | spiv: we also read ".bzr/branch-format" 5 times in the http:// version (and branch/format 6 times)... ugh | 13:57 |
| spiv | jam: I was thinking specifically that you could measure the client's memory consumption with that branch | 13:58 |
| spiv | jam: I already know it is *much* faster | 13:58 |
| maxb | Hi spiv, do you have a thought on when bug 793393 might be addressed? | 14:14 |
| ubot5 | Launchpad bug 793393 in Ubuntu Distributed Development "Please upgrade jubany's bzr-builddeb" [Undecided,New] https://launchpad.net/bugs/793393 | 14:14 |
| jam | spiv: copying the branch locally, 'time bzr branch --stacked bzr://localhost/..." is then 181MB and 1m38s. "time bzr-spive branch --stacked" is 1m26s and 134MB peak memory. | 14:16 |
| spiv | maxb: oh right. I'll do it first thing tomorrow morning my time unless you convince someone else to do it first :) | 14:18 |
| maxb | thanks :-) | 14:19 |
| spiv | jam: hmm, less of a time win on this branch than another test case I have, but not too shabby. (Especially as it has to pay the costs to insert the data into the repo and so validate it where stock bzr doesn't) | 14:19 |
| maxb | Once that's done, and assuming the importer branch itself is up to date, sysvinit can be requeued, and I hope should work | 14:19 |
| spiv | jam: And the memory improvement is good news :) | 14:20 |
| spiv | jam: thanks! | 14:20 |
| jam | spiv: this is also over the loopback, so bandwidth type stuff doesn't really matter. | 14:34 |
| jam | and you seem to be fairly out of date vs bzr.dve | 14:35 |
| jam | you don't have my groupcompress changes, for example | 14:35 |
| jam | spiv: And I should note: "transferred" changed from 95MB to 32MB | 14:35 |
| jam | so for anything network bound (like most things) this is a massive improvement | 14:35 |
| spiv | It was mainly roundtrips I wanted to improve, happily that approach seems to help lots of other things too. | 14:41 |
| fritz[] | poolie: you still there? | 14:46 |
| spiv | fritz[]: unlikely at this hour | 14:48 |
| fritz[] | sorry, can anyone else help? I'm doing research for one bzr issue and I've modified lockdir.py source | 14:49 |
| fritz[] | now trying to recompile my bzr | 14:49 |
| fritz[] | (on windows) | 14:49 |
| fritz[] | i've done | 14:49 |
| fritz[] | python setup.py install --home ~ | 14:49 |
| fritz[] | without problems | 14:49 |
| fritz[] | but where's my bzr.exe? :) | 14:49 |
| fritz[] | I've got \bin directory with bzr and bzr.bat | 14:50 |
| fritz[] | :( | 14:50 |
| spiv | fritz[]: building the windows installer for bzr is a pretty fiddly job | 14:54 |
| fritz[] | so I see :( | 14:54 |
| spiv | fritz[]: as jam could tell you | 14:55 |
| spiv | fritz[]: but you can run bzr from source, I suspect that's adequate to diagnose your issue | 14:55 |
| jam | fritz[]: I would recommend running from source, and just using "py setup.py build_ext -i | 14:55 |
| jam | which builds "in-place" | 14:55 |
| fritz[] | ok, trying exactly that now .. | 14:55 |
| fritz[] | sec | 14:55 |
| jam | the .exe is much more complex to build. From what you wrote, you can probably run it from the 'bzr.bat' | 14:56 |
| jam | which runs the 'bzr' script using your installed python | 14:56 |
| fritz[] | ok, I've done python setup.py build_ext -i without error | 14:57 |
| spiv | (It *may* be possible to twiddle the library.zip or something of the bzr installed by the official installer, but that will be pretty fiddly and I don't know the exact details) | 14:57 |
| fritz[] | ok, so I should use build\scripts-2.7\bzr now? | 14:58 |
| fritz[] | grr | 14:59 |
| fritz[] | ImportError no module named site | 14:59 |
| fritz[] | ignore :) | 15:03 |
| fritz[] | got it | 15:03 |
| workthrick | jelmer: hiya, around? | 16:36 |
| === deryck_ is now known as deryck | ||
| jelmer | workthrick, hi | 17:07 |
| workthrick | jelmer: looking into writing the mapping now | 17:08 |
| jblue | Hi all, I was looking at this wishlist request: https://bugs.launchpad.net/bzr/+bug/681792 and was curious if something like this has been implemented yet? | 17:14 |
| ubot5 | Ubuntu bug 681792 in Bazaar "wishlist: bzr pull --overwrite-tags" [Wishlist,Confirmed] | 17:14 |
| jelmer | jblue, as far as I know nobody has looked into that yet | 17:15 |
| jblue | what would be be the best practise then, to use --overwrite to sync the tags? | 17:15 |
| jelmer | jblue, there isn't really an easy way at the moment | 17:18 |
| jelmer | jblue, a workaround would be to remember the current revision tip ("bzr log --show-ids --limit 1"), then "bzr pull --overwrite" and then restore the tip | 17:18 |
| jelmer | ("bzr pull -rrevid:<revid> .") | 17:18 |
| jblue | jelmer: in essense, overwriting tags, but not pulling in new revisions | 17:22 |
| jelmer | jblue, that's what the workaround would do | 17:22 |
| jblue | sounds fair, thanks jelmer | 17:22 |
| jelmer | jblue, sorry there isn't an easier way (yet) | 17:22 |
| workthrick | jelmer: do you see any obvious problems (besides performance impact, of course, but we know it's there already) by mapping revisions with multiple parents to id concatenating all of the parent IDs in the lexicographical order? | 17:25 |
| workthrick | hmm | 17:25 |
| workthrick | jelmer: that would result in files changing their IDs on the merge revision, wouldn't it? | 17:26 |
| workthrick | jelmer: also I see BzrGitMappingExperimental there, doesn't it do what I want already? | 17:27 |
| jelmer | workthrick, mapping in way? Using their lexicographically sorted parent ids to generate their id? | 17:28 |
| workthrick | yes | 17:28 |
| jelmer | what if two revisions (with different contents) in the same repository have the same parents? | 17:28 |
| workthrick | jelmer: ah, no, sorry, I wouldn't be touching the _revision_ ids, I just want file ids | 17:29 |
| jelmer | workthrick: in that case I'm not sure what you mean by parents? | 17:29 |
| jelmer | workthrick, files have just one parent directory, usually :) | 17:30 |
| workthrick | jelmer: to make them stable, I decided to give files IDs consisting of "sha1 of first revision in history ever" + path | 17:30 |
| jelmer | workthrick, bzrgitmappingexperimental is intended to preserve the file ids if they're imported from bzr, it stores them in a custom file in the tree. | 17:30 |
| workthrick | but that obviously breaks if you merge two timelines | 17:30 |
| workthrick | ah, so it goes the other way around | 17:31 |
| workthrick | jelmer: so I was thinking of a simple mechanism to make it work for all possible graphs | 17:31 |
| workthrick | jelmer: so actually, what I _think_ should work would be to preserve the ID of the file if it had one assigned before the merge, and otherwise pick the lexicographically smaller parent sha1 on each merge point until you reach a root | 17:34 |
| workthrick | root being a root revision now, not tree root | 17:35 |
| jblue | jelmer: thanks, looking forward to seeing that tags issue fixed. The work-around can be a bit misleading, since if you do a 'pull --overwrite', and there are no changes in revisions, it says "No revisions to pull" and it seems like nothing was changed when it actually has (tags overwritten). | 17:36 |
| jelmer | jblue, yeah, there's a separate (higher priority) bug open about that | 17:36 |
| workthrick | jelmer: obviously it'll have an impact on the performance, but bzr-git caches the mappings, right? So it'd only happen once for each file, possibly with a trivial refresh for every file after a merge | 17:36 |
| jelmer | workthrick, different file ids means it won't be possible to easily merge those kinds of files with bzr though | 17:38 |
| workthrick | jelmer: howso? The ID prefix would only change if you merge a completely unrelated tree | 17:38 |
| jelmer | workthrick: yes, but that means that if you ever merge an unrelated tree, you won't be able to sanely merge from things derived from that tree beyond your merge | 17:40 |
| workthrick | jelmer: that should be covered by "check if the file can be traced to before the merge point", and if so, it'd override the "pick the lexicographically smallest parent prefix" fallback | 17:41 |
| jelmer | workthrick: ah, ok - I missed that | 17:42 |
| jelmer | workthrick: I can't poke any holes in that :) | 17:42 |
| workthrick | jelmer: the only case I can see that'd really fail would be if I have a timeline A, then merge it with B which has a smaller prefix, so it gets picked for newly added files, then I add 'foo' and try to merge A' derived from A which also has added 'foo'. But that'd generate a conflict anyway | 17:42 |
| workthrick | jelmer: ah, good. Now if bzr-git caches that for me without too much work on my side, I should be hopefully not that far away from having a stable and unique mapping :) | 17:43 |
| workthrick | jelmer: as for the technicalities of my plugin, I should just say import bzrlib.plugins.git to be able to use its APIs, right? | 17:44 |
| jelmer | workthrick, yeah, "from bzrlib.plugins.git import ..." | 17:50 |
| workthrick | anything that's suitable for a lazy_import? | 17:50 |
| jelmer | workthrick, depends on how you're going to use it | 17:51 |
| workthrick | OK, I'll just do it the easy way for now, and see what can be optimised later on | 17:51 |
| workthrick | jelmer: hmm, now how can I get at the bzr's idea of what the file is in generate_file_id(self, path)? | 18:45 |
| workthrick | just a path is insufficient | 18:46 |
| === med_out is now known as med | ||
| === med is now known as medberry | ||
| jelmer | workthrick, you have to patch the interface to take more information | 19:34 |
| workthrick | jelmer: aha. Do I need to do it at the bzr-git level, or bzrlib.foreign level? | 19:46 |
| jelmer | workthrick, bzr-git, I don't think this is a bzrlib.foreign method | 19:47 |
| jelmer | as the requirements are different per foreign vcs | 19:47 |
| gour | do you recommend bzr-svn trunk or 1.0.4 with bzr-2.3.3? | 20:20 |
| workthrick | gour: trunk is a bad idea with released bzr versions, since trunk moves to requiring bzr-dev as soon as it gets the chance | 20:28 |
| gour | workthrick: i recall having some interface-version problem (on freebsd)...let me try with 1.0.4. | 20:29 |
| gour | yeah, bzr plugins complain: "Unable to load plugin 'svn'. It requested API version (2, 2, 0) of module <module 'bzrlib' from '/usr/local/lib/python2.7/site-packages/bzrlib/__init__.pyc'> but the minimum exported version is (2, 3, 0), and the maximum is (2, 3, 3)" | 20:31 |
| workthrick | gour: that means you need an earlier version of bzr-svn | 20:58 |
| workthrick | ah wait, no | 20:58 |
| workthrick | a later one | 20:58 |
| workthrick | the releases should be tagged with what bzr's they work with | 20:58 |
| gour | heh...that's the latest official release | 20:59 |
| workthrick | then grab an unofficial one, but not the trunk | 21:00 |
| gour | supposed to work with 2.2 | 21:00 |
| workthrick | since trunk is a moving target, and will probably require 2.4 at this point | 21:00 |
| jelmer | trunk supports 2.3 and 2.4 | 21:00 |
| gour | ok...let's try trunk then | 21:00 |
| workthrick | huh, is bzr-standalone for win32 supposed to be able to use paramiko? | 21:01 |
| workthrick | because I just updated from 2.2/py to 2.4b3/standalone, and it broke | 21:01 |
| workthrick | bzr: ERROR: Unrecognised value for BZR_SSH environment variable: paramiko | 21:01 |
| === yofel_ is now known as yofel | ||
| gour | it's ok now (bzr-svn) | 21:02 |
| workthrick | gour: btw, bzr tags -d lp:bzr-svn is useful for seeing what versions are tagged | 21:18 |
| gour | workthrick: thanks...i had a trust in freebsd ports which says that 2.3.3 is ok | 21:19 |
| === medberry is now known as med_out | ||
| mwhudson | jelmer: i really hope you've been using scripts to do you code import mangling | 22:58 |
| poolie | hi all | 23:02 |
| jelmer | mwhudson, nope, it's all me | 23:30 |
| mwhudson | jelmer: you are a very determined man, clearly | 23:30 |
| jelmer | mwhudson, I managed to convert all the failing svn-via-cscvs imports, hopefully we can get to a point where failing imports can become OOPSes | 23:30 |
| jelmer | anyway, there's not much I can do anymore now that Launchpad's down and it's already been a way too long day | 23:32 |
| jelmer | ttyl :) | 23:32 |
| mwhudson | jelmer: it would be nice to be able to distinguish "the remote server went away" from some of the other problems | 23:32 |
| mwhudson | jelmer: bye | 23:32 |
| jelmer | mwhudson, we can just catch a handful of bzr errors that are indications of user errors | 23:32 |
| mwhudson | cool | 23:33 |
| poolie | jelmer: maybe you should write an api script? | 23:52 |
| poolie | it might make it easier next time? | 23:53 |
| poolie | i don't know | 23:53 |
| jelmer | poolie, I'm hoping to have Launchpad do it for me next time :) | 23:55 |
| jelmer | the cscvs imports should be the only ones where it's impossible to reliable tell whether it's failing because of a bug or circumstances (repository removed, etc) | 23:55 |
| poolie | fair enough | 23:57 |
| poolie | having lp do it automatically would of course be far better | 23:57 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!