poolie | hi all | 00:09 |
---|---|---|
mathrick | hiya poolie | 00:17 |
mathrick | poolie: <mathrick> re: bug #838469, should improvements to mini-tutorial be submitted for released versions as well, or will they only be accepted for bzr.dev? | 00:17 |
ubot5 | Launchpad bug 838469 in Bazaar "Mini tutorial doesn't explain repository directories" [Low,Confirmed] https://launchpad.net/bugs/838469 | 00:17 |
poolie | hi mathrick | 00:19 |
poolie | we can take doc updates for released versions | 00:20 |
mathrick | poolie: great, should I then make a patch against the 2.4 braching point, or should I rebase it, or should it rather be cherry-picked for release branches? | 00:22 |
poolie | just do a revision based off lp:bzr/2.4 tip | 00:23 |
mathrick | OK | 00:23 |
vila | hi all ! Happy Friday ! | 06:19 |
AuroraBorealis | yay | 06:20 |
jam | morning all | 06:54 |
=== vila is now known as vila_at_dentist | ||
jbicha | hi, is there already a bug open for bzr-builddeb's lack of support for .tar.xz? that's going to be especially annoying since GNOME 3.3 won't ship other tarballs | 07:28 |
mgz | morning all | 07:35 |
poolie | jam, hi - you're feisty today! :) | 07:58 |
=== vila_at_dentist is now known as vila | ||
jam | mgz: I have some ideas about bug #869366, are you looking at it, or just reporting it for posterity? | 08:25 |
ubot5` | Launchpad bug 869366 in Bazaar "TestTCPServerInAThread.test_server_crash_while_responding random failure" [Medium,Confirmed] https://launchpad.net/bugs/869366 | 08:25 |
mgz | just reporting, as I ran into it EOD yesterday | 08:25 |
mgz | I seem to be getting lucky with random failures on PQM | 08:25 |
mgz | looking at the test, it seems possible for the subthread to not do any futher work after the sync | 08:29 |
vila | mgz: yes, that's the race I was mentioning the other day, the same one exists for the test just above, it triggers very rarely and it took quite a while to put my finger on it | 08:32 |
vila | mgz: the main thread really want to wait for both the connection thread to raise the exception *and* to the server thread to re-raise it | 08:33 |
mgz | I'm not even sure what tools python gives us to resolve such problems | 08:33 |
mgz | I guess in this case try:/raise/finally:/sync2 would work? | 08:34 |
mgz | because the exception won't propogate until the test looks for it? | 08:35 |
vila | mgz: the test looks for the exception to be swallowed :) | 08:35 |
jam | mgz: for that particular test, you could call 'server.stop_server()" instead of "server.pending_exception()" | 08:35 |
vila | mgz: the previous test (test_server_crash_while_responding) looks for the exception | 08:35 |
jam | stop can be called twice | 08:35 |
jam | and that way it will have joined its threads | 08:35 |
vila | which is not what the test is about: test_exception_swallowed_while_serving | 08:36 |
vila | this a test about the server not about the client | 08:37 |
vila | we don't want the server to shut down, we want it to keep serving (swallowed_while_serving) | 08:39 |
jam | vila: that is the other test | 08:39 |
jam | "test_server_crash_while_responding" wants it to raise the exception | 08:39 |
jam | we can just change pending_exception to stop_server and it will sync with the clients | 08:39 |
jam | client threads | 08:39 |
jam | maybe connection threads is a better term | 08:40 |
jam | though I guess that doesn't make the tests symmetric except for the exception being ignored. | 08:40 |
vila | with 'that' == test_server_crash_while_responding, this test is about crashing the server, not shutting it down | 08:42 |
vila | pending_exception is what is used to implement shutting down | 08:42 |
vila | which illustrate why I disagreed with your modifications which were about client behavior and as such replacing these tests instead of adding your owns | 08:43 |
vila | bug mgz encounters the issue in 2.4 so your modifications are out of scope here | 08:44 |
vila | mgz: by the way, thanks for filing that bug, I thought I did as this failure has been seen on babune on several occasions (rare enough to not trigger a fix) | 08:45 |
mgz | hm yes, I might not have said that in big enough letters in the bug, but this was a landing on the 2.4 branch | 08:45 |
vila | mgz: ... and *never* triggered on pqm AFAIK... before you claimed the unlucky badge :) | 08:46 |
mgz | two different random failures in about four landings on 2.4 PQM is pretty good going | 08:49 |
vila | mgz: yup, don't let them go ! :) | 08:49 |
mgz | oh hey, looking at babune, lucid is red | 08:50 |
mgz | and guess what the failing test is :) | 08:50 |
vila | mgz: but can we assume the cause is the same now that the test is not ? | 08:50 |
mgz | ...one thing going for testtools is seeing both tracebacks makes this miles clearer | 08:51 |
mgz | vila: if there's still just the one sync before the raise, then yes I think | 08:51 |
vila | mgz: I was kidding ;) test_server_fails_while_serving_or_stopping shouldn't suffer from the same race because it explicitly call stop_server I think... | 08:54 |
poolie | hi mgz, vila | 08:54 |
vila | helli pooloe | 08:54 |
vila | meh | 08:54 |
vila | hello poolie ;) | 08:54 |
mgz | hellipolis? | 08:55 |
vila | mgz: so the ' Wait for the exception to propagate' is a lie, it waits for the exception to be 'about to be raised' and just doing 'sync.wait()' is not enough to guarantee that. That's where the race is. A possible fix will be to connect with another client and get served properly (i.e. yes, the server is still up and running) | 09:02 |
vila | mgz: trying to be more precise would require hooking somewhere the server really swallows | 09:02 |
vila | mgz: hehe, but that's in fact required as this test server cannot server properly (by test design :) | 09:12 |
AuroraBorealis | is there a way to force bzr to use its built in ssh rather then another ssh that is on the path? | 09:37 |
AuroraBorealis | (installed git, and of course it has an ssh.exe that causes bzr to just hang for some reason) | 09:38 |
jam | AuroraBorealis: "export BZR_SSH=paramiko" | 09:39 |
AuroraBorealis | where do i put that if i'm on windows? | 09:39 |
jam | if you're on Windows it is spelled | 09:39 |
jam | set BZR_SSH=paramiko | 09:39 |
jam | but you can set it in global env vars | 09:39 |
jam | right click My Computer, Properties, Advanced System Settings, Environment Variables | 09:39 |
AuroraBorealis | ah kk | 09:40 |
AuroraBorealis | user or system variable, does it matter? | 09:40 |
jam | AuroraBorealis: doesn't matter for you | 09:42 |
jam | depends if you have more than one user on the machine | 09:42 |
AuroraBorealis | but python getenv can access both ? | 09:42 |
AuroraBorealis | (guess just a python question now =P ) | 09:43 |
mgz | bug 641330 is the shelve issue | 09:45 |
ubot5` | Launchpad bug 641330 in Bazaar "unable to unshelve shelved root-id change" [Medium,Confirmed] https://launchpad.net/bugs/641330 | 09:45 |
mgz | bug 82555 looks like the root cause | 09:45 |
ubot5` | Launchpad bug 82555 in Bazaar "Merging to an empty branch doesn't work" [High,Confirmed] https://launchpad.net/bugs/82555 | 09:45 |
mgz | bug 242175 is the mitigation for that Riddell added for the merge case | 09:46 |
ubot5` | Launchpad bug 242175 in Bazaar "ValueError: WorkingTree.set_root_id with fileid=None when merging into empty branch" [High,Fix released] https://launchpad.net/bugs/242175 | 09:46 |
mgz | AuroraBorealis: windows merges them before starting a process as your user | 09:48 |
AuroraBorealis | k | 09:48 |
AfC | Do bzr-builddeb's `import-upstream` and bzrtools's `import` use the same change detection / file-id (ie, file checksum, perhaps) algorithms? | 10:15 |
AfC | (anyone know?) | 10:16 |
AuroraBorealis | not me :> | 10:16 |
jelmer | AfC: hi | 10:17 |
jelmer | AfC: change against what? | 10:17 |
AfC | jelmer: hi | 10:17 |
AfC | jelmer: so, in both cases, there's considerable work done to detect (for example) renames, right? | 10:18 |
AfC | [so I infer] | 10:18 |
jelmer | AfC: no, no renames are inferred in either case as far as I know | 10:18 |
AfC | oh | 10:18 |
jelmer | I could be wrong, I don't have much experience with either | 10:19 |
jelmer | actually, "bzr import" could help | 10:19 |
jelmer | since it doesn't commit | 10:19 |
jelmer | so you can do "bzr import ..; bzr mv --auto; bzr ci -m 'import x'" | 10:19 |
AfC | jelmer: ok, well, it's doing *something* for the minutes of CPU time its sitting there burning away | 10:19 |
AfC | mv --auto, yeah, that might be interesting | 10:19 |
jelmer | AfC: which one is burning CPU? import or import-upstream? | 10:20 |
AfC | [both] | 10:20 |
AfC | I just now did an import; import-upstream was taking ~9 minutes each. | 10:20 |
jelmer | AfC: what are you importing? | 10:20 |
AfC | :) | 10:20 |
AfC | that large well known project prominently using someone else's version control system that I've been blundering along with this p | 10:21 |
AfC | past week | 10:21 |
AuroraBorealis | if its git, we are already seeing that fast-import or whatever is taking huge amounts of time / memory. | 10:22 |
AfC | AuroraBorealis: so I'm not working with bzr-git in this case; I've instead used bzr-builddeb's facilities against the upstream project's release tarballs. | 10:23 |
AuroraBorealis | ah | 10:23 |
jelmer | AfC: I don't think either import or import-upstream have been optimized for kernel-sized trees.. | 10:23 |
jelmer | AfC: any particular reason you're not using the launchpad imports, or are you just seeing how far you can get? | 10:24 |
AfC | jelmer: that's ok, I'm not bitching about import performance per se | 10:24 |
AfC | jelmer: so, having established the deterministic behaviour, I tried the lp imports branch | 10:24 |
AfC | jelmer: but I kept getting swap thrash. I'm sure I'll be able to make use of it at some point, though | 10:25 |
jelmer | AfC: you're getting swapping trash even just running "bzr branch lp:linux linux" ? | 10:26 |
AfC | yeah | 10:28 |
AfC | [anyway, that's done, I let it run overnight or whatever] | 10:28 |
jelmer | AfC: what version of bzr are you running? | 10:28 |
AfC | jelmer: daily | 10:28 |
AuroraBorealis | is lp:linux already a bazaar repo? | 10:28 |
AfC | yes | 10:28 |
AfC | AuroraBorealis: ^ | 10:29 |
AuroraBorealis | weird that its thrashing when its just branching. | 10:30 |
AfC | it just took a long time; it was quite a few days ago; I've had bigger problems since then doing other stuff, but as I said, it's here now should I need it. | 10:30 |
AfC | bzr tags | 10:31 |
AuroraBorealis | might be related to the uhh | 10:31 |
AuroraBorealis | bug i'm having when importing the git linux repo | 10:31 |
AuroraBorealis | where its using so much memory (therefore thrashing if you don't have enough ram) cause its keeping the entire history in memory | 10:32 |
jelmer | AuroraBorealis: where are you getting that? using bzr-git, bzr-fastimport, regular clone? | 10:35 |
jelmer | none of them should really be keeping all history in RAM | 10:36 |
AuroraBorealis | https://bugs.launchpad.net/bugs/864217 | 10:37 |
ubot5` | Ubuntu bug 864217 in Bazaar "MemoryError when repacking repository with large numbers of revisions " [Medium,Confirmed] | 10:37 |
AuroraBorealis | doing fast-import of the git linux kernel uses about 2 gigs of memory , and it wasn't even done | 10:37 |
AfC | jelmer: well it is attempting to keeping 3.0 GB of something in memory. This doesn't go down well :/ | 10:37 |
AuroraBorealis | and mgz said something about it was keeping history in memory | 10:39 |
AfC | jelmer: (this was even with bzr branch -r 1 URL; I never could get it to finish with -r > 100) | 10:39 |
AfC | (I imagine looping one revision at a time over n days would have worked out, but having established it was deterministic and didn't have missing revision relative to the lp import, I knew I could trust that) | 10:40 |
jelmer | AfC: did you see the bug I filed earlier about -r1 not actually doing that? | 10:46 |
mgz | jelmer: from a (partial) memdump AuroraBorealis had from fast-import, there was a _KnownGraph taking up a bg chunk of memory | 10:48 |
mgz | that may well be a bug, there's a _known_heads cache dict that seems essentially unbounded | 10:48 |
mgz | it only ever gets cleared on a couple of error paths | 10:49 |
AuroraBorealis | have you worked on meliae not actually dumping all the objects? | 10:49 |
AuroraBorealis | so we can see more of whats in the dumps? | 10:49 |
mgz | nope, but I've got that down as a project for this weekend | 10:49 |
AuroraBorealis | kk | 10:50 |
AuroraBorealis | i should be around, although you seem to be on the other side of the world from me so hours will be interesting =P | 10:51 |
mgz | on Monday I saw a reproducable segfault, some type errors, and the issue with missing objects causing the dump to not load | 10:52 |
mgz | fixing all those should get us somewhere, but the incomplete dump may still be a win64 size/alignment issue which will be a pain to work out from here | 10:53 |
AuroraBorealis | oh gee ;< | 10:53 |
AuroraBorealis | do you have access to a win64 computer? | 10:53 |
mgz | no, that's the fun bit :) | 10:53 |
mgz | the code has the wrong size of all the pointer->int casts though, so there's some hope I can just fix things till it magically works | 10:54 |
AuroraBorealis | i'll mail you my crappy 500 dollar laptop with windows 7 64 bit on it thats collecting dust behind me | 10:57 |
=== Vorpal_ is now known as Vorpal | ||
AfC | jelmer: no, I didn't; I'll look | 11:00 |
mgz | where on the other side of the world are you AuroraBorealis? :) | 11:00 |
AuroraBorealis | arizona, usa | 11:00 |
AfC | I would have thought you would have to go a bit further north than that to see the aurora borealis. Unless we just got spanked by a giant gamma ray burst... | 11:01 |
fullermd | Plate tectonics. Just takes a little longer. | 11:02 |
AfC | ah, yes, of course :) | 11:02 |
AuroraBorealis | yes | 11:03 |
fullermd | ('course, it does make you wonder what kinda nutcase would be in AZ, awake, and on IRC at this hour :p) | 11:03 |
AfC | [I only ever saw it once. I was rather far up, as I recall. Was pretty] | 11:03 |
AuroraBorealis | the nutcase who was procostinating on his small prolog assignment | 11:03 |
* mgz thinks fullermd isn't one to talk | 11:03 | |
fullermd | Saying 'prolog' isn't helping your case! | 11:03 |
AuroraBorealis | haha | 11:04 |
fullermd | Hey, I'm a solid hour ahead of him. That makes me totally sane. | 11:04 |
AuroraBorealis | haha | 11:07 |
AuroraBorealis | anyway, bed time for me. school in a few hours | 11:11 |
AuroraBorealis | chat with you this weekend i guess mgz | 11:11 |
=== medberry is now known as med_out | ||
=== beuno is now known as beuno-lunch | ||
jml | james_w: would you mind merging my branch? | 16:05 |
james_w | sure thing | 16:10 |
vila | jml, james_w : sry about that, I planned to do it but get interrupted longer than expected | 16:28 |
jml | vila: np | 16:29 |
=== med_out is now known as medberry | ||
=== beuno-lunch is now known as beuno | ||
=== deryck is now known as deryck[lunch] | ||
=== cody-somerville_ is now known as cody-somerville | ||
=== yofel_ is now known as yofel | ||
=== deryck[lunch] is now known as deryck | ||
sponge- | is there a way to do a sparse/bare checkout in bzr? in svn i'd do --depth immediates and then svn up --set-depth infinity on the folders i wanted, looking for something similar | 19:22 |
elmo | when merging someone else's branch, should I commit, then do fix ups, or can I merge, fix + commit result and still be able to usefully differieniate the two sets of changes? | 20:19 |
lifeless | elmo: difftastic/difftacular - I never remember the name - can untangle them for you | 20:36 |
lifeless | so sure, do whatever ;) | 20:37 |
wgz | lifeless: can I pester you for your thoughts on bug 866107 at some point? | 21:45 |
ubot5 | Launchpad bug 866107 in Bazaar "Use testresources for bzr selftest" [Medium,Confirmed] https://launchpad.net/bugs/866107 | 21:45 |
lifeless | sure, not right now sadly | 21:48 |
wgz | I shall ping again over the weekend at some point. | 21:48 |
wgz | and in other news, I have once again made PQM fail in some weird way | 21:53 |
wgz | this time... all the tests pass but... | 21:53 |
wgz | star-merge succeeded at Fri Oct 7 16:26:04 2011 (0:24:02.142211) | 21:54 |
wgz | 'Exception processing merge: [Errno 2] No such file or directory' | 21:54 |
vila | O.o | 21:55 |
wgz | stderr has the slightly suspicious: | 21:56 |
wgz | Fri Oct 7 16:02:43 UTC 2011 : selftest starts | 21:56 |
wgz | failed to open trace file: [Errno 13] Permission denied: '/you-should-use-TestCaseInTempDir-if-you-need-a-log-file' | 21:56 |
wgz | Fri Oct 7 16:26:01 UTC 2011 : selftest ends | 21:56 |
wgz | but there's no mention of a test with that issue in the test suite output | 21:56 |
hazmat | is there a cli command to get the latest revid out of a repo... ie. analogue of revno? | 21:58 |
vila | red herring, it's a leak somewhere, has been there for ages | 21:58 |
vila | wgz: ^ | 21:58 |
wgz | okay, so it's the merge failing somehow that's the issue | 21:58 |
wgz | but... after all the tests run, which is just weird | 21:58 |
vila | wgz: more precisely some TestCase should be a TestCaseInTemp... err, yeah, that's what the path says ;) | 21:59 |
wgz | hazmat: `bzr revision-info`? | 21:59 |
vila | wgz: could be related to the issue with the lack of mails to bazaar-commits | 21:59 |
wgz | I'll post to the mp so MvG knows I'm not just holding out on him. | 22:00 |
lifeless | wgz: anyhow, testresources is good; I plan to supercede its model with testfixtures once I or someone else implements a fixture graph with reasonable introspection semantics (such that you can do the testresources planning on unsetup fixtures) | 22:03 |
wgz | lifeless: do you have any particular opinions about what bzr needs on this front? | 22:12 |
lifeless | wgz: I'd write your stuff as fixtures and use a shallow fixture->resource-manager shim | 22:12 |
lifeless | as I don't expect you to need deep route planning, merely aggregation of all the tests using a given external server | 22:13 |
wgz | one part of your fixtures readme really confused me | 22:14 |
wgz | resources had a clear part about dependent resources | 22:15 |
hazmat | wgz, perfect thanks.. didn't show up on a bzr help commands | grep rev | 22:15 |
wgz | whereas fixtures had a long example about how you couldn't share a tempdir if there were also a db and a webserver | 22:15 |
wgz | which... I didn't get at all | 22:16 |
lifeless | fixtures is a simpler model | 22:17 |
wgz | but that seems simpler to the point of not being useful | 22:18 |
lifeless | fixtures has seen more use in less time | 22:18 |
lifeless | this is one of the paradoxes of design :) | 22:18 |
wgz | if I can't share the tempdir created by TestCaseInTempDir (with some added carefulness in reset), I can't build any other fixtures of use on top of it | 22:18 |
lifeless | of course you can, if you want to | 22:19 |
wgz | nearly everything thats expensive enough to want to share between tests first derives from TestCaseInTempDir | 22:19 |
wgz | does testresources basically get around this by running all webserver tests together, then all db tests, and if it can't do that tearing down the lot between? | 22:20 |
lifeless | well back up a step | 22:22 |
lifeless | I think you have some conflation | 22:22 |
wgz | this is likely. | 22:22 |
lifeless | TestCaseInTempDir using unique temp dirs for each test case | 22:22 |
wgz | it shares a ROOT_DIR | 22:22 |
wgz | this is a bad thing, because it's global state | 22:22 |
lifeless | yes | 22:22 |
lifeless | its something we were working to nuke | 22:22 |
wgz | this is something I'd like to nuke with one of your nice packages :) | 22:23 |
lifeless | anyhow, fixtures describes an extended context manager basically | 22:23 |
lifeless | which happens to be very useful for tests because it exposes things like getDetails | 22:23 |
lifeless | and a convenient addCleanup for the code within it. | 22:23 |
lifeless | testresources describes an optimisation framework for expensive things; each of which could be a fixture (but needs a graph which fixtures doesn't expose) | 22:24 |
lifeless | the fixtures README talks about using just fixtures in similar situations to what testresources is targeted at - and in that world its kludgy, because of the lack of an introspectable graph | 22:25 |
lifeless | (you can't tell that the webserver and db are going to be using the same temp dir, and you can | 22:25 |
lifeless | 't (if they use addFixture) cleanup just the webserver or just the db, because that will implicitly cleanup the tempdir, hurting the other user | 22:26 |
lifeless | so | 22:26 |
lifeless | this is why I was saying to use testresources, but write the resources as a manager shimming across to a fixture | 22:26 |
lifeless | now, as far as test dir reuse, I think you want an external server to have the following properties: | 22:26 |
lifeless | - totally decoupled from the test process | 22:28 |
lifeless | - but managed by it | 22:28 |
lifeless | so I don't think you want testcaseintempdir to have anything to do with the server | 22:29 |
lifeless | just have the server make its own temp dir, for its logs etc, and each test case can tell the server where the test case is rooted etc | 22:29 |
wgz | well, often the tests that need a server have their own local branch for the client | 22:29 |
wgz | which then chats to the server | 22:29 |
wgz | so, it would have a server resource, and some other resources for the actual test | 22:29 |
lifeless | definitely a server resource, but unless you intend to share the other resources, they don't need to be exposed to testresources | 22:30 |
wgz | I think it would be worth sharing the safe-contain-in-filesystem infrastructure | 22:30 |
lifeless | and can be just fixtures (which bzr already uses though without using the library) | 22:31 |
wgz | the branch and trying to escape test isolation stuff | 22:31 |
wgz | which is basically just some state on disk, that doesn't change unless something goes wrong (and could then be reset) | 22:31 |
lifeless | sure | 22:31 |
wgz | fixtures does have the advantage of not needing the actual package | 22:32 |
wgz | and bzr may not really need the fancy graph stuff | 22:32 |
lifeless | I think bzr should use the packages :) | 22:32 |
lifeless | the fixtures package has very useful helpers | 22:32 |
wgz | the existing test arrangements may be enough without whole test suite reordering | 22:32 |
wgz | as much as I like your graph code :) | 22:32 |
lifeless | and the testresources graph stuff will be needed if you want a single reused server for each of the server types | 22:32 |
wgz | well, we really only need it shared 'enough' | 22:33 |
lifeless | bzr tests are currently totally isolated except for the safety mechanisms | 22:33 |
lifeless | wgz: I -really- wouldn't do 'enough' - do it right, don't NIH | 22:33 |
wgz | starting a new one per module rather than per test method is still a win. | 22:33 |
lifeless | wgz: with the in-test servers, thats not at all clear when compared to an out of test server | 22:34 |
lifeless | wgz: starting in-test servers is pretty darn cheap | 22:34 |
wgz | yeah, I feel very fearful of the NIH, but I'm also wondering if I'm going to need as much plumbing as doing something dumb like that would be anyway | 22:35 |
wgz | I think I actually like the testresources api better (except for that ugly hack you have at the bottom to sniff out a TestResult), but if leaning towards fixtures I guess I need to work out how they could actually be used instead | 22:38 |
wgz | ...I don't really like context managers with unittest, doesn't fit with the seperate methods for setup/teardown style | 22:39 |
wgz | +you | 22:40 |
wgz | in one of those malformed sentences somewhere | 22:40 |
wgz | okay, thanks for your time lifeless, I'll try some things out and see where I get | 22:42 |
lifeless | wgz: so my point was you need two things, you should use both of them | 22:44 |
lifeless | testresources takes care of the ordering and external bringup-takedown aspects | 22:45 |
lifeless | fixtures takes care of easy code for encapsulated services | 22:45 |
lifeless | wgz: tearDown should be deleted, can't be for compat, but don't use it. | 22:45 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!