[01:51] poolie: http://www.network-theory.co.uk/cgi-bin/ghm.cgi? [02:23] lifeless: hey, perhaps you can help with something [02:23] http://bazaar.launchpad.net/~agilence/+junk/tonomundo/changes shows last revision is 192 [02:24] http://bazaar.launchpad.net/~agilence/+junk/tonomundo/.bzr/branch/last-revision shows last revision is 13 [02:24] any idea what causes this? [02:25] that's odd [02:25] I've seen it happen before [02:25] but I can't remember why, or what we did to fix it [02:25] i think we had some bugs in the past where the cached last revno could be wrong [02:25] it's possible that pushing using a buggy client could get it wrong [02:25] is there a bug for this, or is it causing a crash? [02:26] bug 599492 [02:26] Launchpad bug 599492 in Launchpad Bazaar Integration "Incorrect links to revisions (affected: 1, heat: 6)" [Undecided,New] https://launchpad.net/bugs/599492 [02:26] lifeless, does that ring any bells with you? [02:28] spiv^^ [02:28] hey there, what's up with nested trees/submodules support? I see https://blueprints.launchpad.net/bzr/+spec/nested-tree-support being dead for 4 years! [02:29] nekohayo, hi, i would recommend you use scmproj [02:29] http://doc.bazaar.canonical.com/plugins/en/scmproj-plugin.html [02:29] poolie: is that the official answer now? [02:30] poolie, actually I'm asking this because I'm dying having to use git to pull pitivi... I'd like to be able to use bzr-git [02:30] poolie: what about submodule support for bzr-git? [02:30] thumper, that's not on our plate at the moment [02:30] * thumper nods [02:30] poolie: no bells other than "this used to happen sometimes in the past" [02:30] I've asked on the bug about the client bzr version [02:30] 'bzr reconcile' ought to fix it. [02:31] is it on some not-so-far roadmap or ETA, or should I not expect this within years? [02:31] thumper: hi, sorry [02:31] (I can't provide a patch, sadly) [02:31] nekohayo: I think decisions were made at a recent sprint to allow nested trees to work as a plugin initially [02:32] yes [02:32] nekohayo: to clear the road for some development on it [02:32] thumper: so I think that that indicates a 'wrong' last-revision file in the branch. [02:32] nekohayo: but I don't know if anyone is moving with it right now [02:32] lifeless: yes... [02:32] thumper: if the user runs 'bzr reconcile' bzr will rewrite it. [02:32] lifeless: and push again? [02:32] lifeless: or we should reconcile the LP version? [02:33] push --overwrite (to force it to update) [02:33] or reconcile, and the next push that actually does stuff should fix it automatically. [02:33] you could reconcile the LP version too [02:33] can you reconcile a remote branch? [02:34] yes but its a tad slow [02:36] The user will probably need to reconcile their local copy too. [02:41] thanks [02:41] bug has been updated [02:43] thumper: nested trees - marius kruger is interested in running with it, but hes out of cycles [03:16] spiv, in the 2.2.0 milestone page, how many bugs does it say under "assigned to you" [03:16] the actual field, not adding it up yourself [03:16] https://edge.launchpad.net/bzr/+milestone/2.2.0 ? yes fail. [03:17] using caching is hard. [03:17] it seems like in principle you could warn about that [03:17] Vary [03:17] In general we want Vary on everything that is cachable. [03:18] Because there are so many...variations :P [03:19] mm [03:29] thumper: bzrlib/tests/blackbox/test_serve.py [03:30] lifeless: ta [03:30] 12 tests [03:30] 336 lines; balance is ok but its a lot of complexity [04:01] poolie: hi [04:13] argh [04:13] so I have a branch, which I've committed several revisions to [04:14] and now I realise that in one of the earlier revisions, I accidentally removed a file and readded it elsewhere, instead of using bzr mv [04:15] is there an easier way to fix the branch up besides creating a new branch and manually replaying the revisions on this one, one at a time? [04:22] hmm, I'm going to try making a copy of the branch, uncommitting past the faulty revision, fixing it up, then rebasing the complete branch on top of that one === jfroy_ is now known as jfroy [04:28] lifeless: it seems that push --overwrite didn't update the last revision: https://bugs.edge.launchpad.net/launchpad-code/+bug/599492/comments/3 [04:28] Launchpad bug 599492 in Launchpad Bazaar Integration "Incorrect links to revisions (affected: 1, heat: 6)" [Undecided,Incomplete] [04:33] argh, now I have a "contents conflict" [04:35] * idnar continues hurtling down the rabbit hole [04:38] okay, I think that worked [05:10] thumper: thanks === FryGuy_ is now known as FryGuy- === mlh_ is now known as mlh [07:18] vila: ping [07:33] hi all ! [07:33] lifeless: pong [07:34] hi there vila [07:34] lifeless: do GlobalConfig and LocationConfig objects sound like good candidates for bzrlib.state ? [07:34] how's it going? [07:36] poolie: fine, I encounter funny details while working on the config file locks, like: lock info trying to access the config while locking it and lock trying to create the config dir to access a non-existent config file but I'm almost finished [07:38] vila: hi, was looking for a 'ForceNeeded' error but couldn't see one [07:38] vila: uhm, *if* they are currently getting globally memoised, sure. [07:39] vila: but I wouldn't add anything to it that isn't already globally memoised. [07:39] vila: I think we should have smaller lifetime context objects for such things. [07:41] lifeless: the context here is that to fix the bug I need to lock write access but also read access which makes accessing the config more costly. On the other hand we actually almost always access the file for every config variable. The file is read/parsed and all (variable/value) tuples are indeed memoised... to be thrown away immediately [07:41] vila: why do you need to lock read access? [07:42] vila: that smells very wrong to me [07:42] lifeless: since there is a very high suspicion (I couldn't get a corrupted conf file yet), it means two writers have been able to write to the same file at the same time. I see no reason to believe that a reader can't get a partial file while a writer is producing it [07:43] vila: I think it would be a lot better to do the following: [07:43] - ensure that the file is written with our atomic-write facilities [07:43] it is [07:43] e.g. transport.put_bytes() [07:43] well, no, it isn't [07:43] in which case readers cannot get partial files [07:44] and [07:44] - ensure that changes to it do not do so from stale data - changes should be 'lock, read, write, unlock', never 'read, lock, write, unlock' [07:44] that's already done, with tests [07:44] - do that by making configs gotten without locking be immutable. [07:45] so that we're not dependent on tests for correctness. [07:45] the third point is guaranteed from the second one [07:45] vila: no its not :) [07:45] well, at least from the config object point of view, extending this rule to callers.... [07:45] vila: 1 word: plugins [07:46] OTOH you may have done 3 already - I haven't seen your code. [07:46] anyhow, if you have those three things, why do you need to lock when reading? [07:46] locking is /very/ expensive and there is no way we should consider for more than an msec doing that on every execution of bzr [07:46] I don't have the first yet [07:46] it would also cause all sorts of bugs on readonly filesystems. [07:46] vila: well, thats easy then :) [07:47] config dir on read-only FS.... missed that one, not sure it's easy to achieve but no reason to make it harder [07:47] urk, yes, we should really not lock on read [07:49] yet, even if the write is atomic, do we rely on the FS to keep providing the file content as it was when opened even if the path is no longer valid ? [07:51] vila: there are a few cases. I think its ok. Its certainly no worse than today, right ? [07:52] lifeless: no worse yes, at least we cover the concurrent writers which the bug seems to be about [07:52] right. [07:54] lifeless: regarding point 3 above (immutable), if you keep your config objects around, even if immutable, they can be come out of date, sharing them is a way to address the problem hence my question about state [07:54] s/be come/become/ [07:54] vila: different problem [07:54] vila: that can happen today. [07:54] lifeless: yes [07:54] vila: and I don't see any need to change any code beyond config.py for this patch; we want to fix 'concurrent writers', nothing less, nothing more. [07:55] The reason to focus on the use case here is because we are generally happy with configs I think [07:55] lifeless: I don't intend to fix more than needed, just discussing the problem while it's fresh [07:55] there aren't other issues to tackle, unless we make them ;) [07:56] lifeless: and also checking I understand how you intend bzrlib.state to be used ;) [07:56] s//to use/ [07:56] vila: I want to delete it totally [07:56] s!s///!! [07:56] Unsupported or unknown search engine! [07:56] vila: but its a tool to transition existing global state from all over the library into one place. [07:56] vila: which is better than the same state spread out. [07:57] config files are global state, outside of the code currently, but still a global state [07:57] vila: unless they are assigned to a global variable, they aren't for the purpose of this discussion. [07:57] import universe is cheating! [07:57] then I'm done :) [07:58] cool [08:06] using threads to test concurrent actors is yummy, I even just found away to avoid hangs (at least some) when the test fail... [08:06] gee, that's typo day ! s/away/a way/ [08:06] freudian slip for get awat may be... [08:07] away of course, couldn't avoid the typo there, story of my life :) [11:28] Hi guys. I have a commit that introduced some good and bad changes on a file. What would be the best way to 'unpick' this file for a new commit? [11:28] CaMason: bzr shelve - is your friend. [11:29] yes, I figured that would help [11:31] so, should I get the copy of the file before the bad commit? === nlisgo_ is now known as nlisgo === Chex_ is now known as Chex [16:21] CaMason: you may be able to uncommit and then shelve. [16:22] bzr uncommit takes off the last commit and restores to that state IIRC [16:53] bzr: ERROR: No module named _curses [16:53] i'm on CentOS and CANNOT find th epython curses module [16:58] vila: hey [16:58] vila: your lp:~vila/bzr/525571-lock-bazaar-conf-files branch has conflicts [16:58] tsmith: This should be provided by the Python installation itself, i.e. your Python installation is broken [16:59] well that's good to know! [16:59] i couldn't find source code for pycurses anywhere [17:05] jelmer: wow, I was about to tell you I submitted this :) [17:06] :-) [17:06] jelmer: as far as bzr-svn is concerned, it should just me a matter of requiring 2.2 and changing the base class for subversion.conf [17:06] ah, great [17:07] vila: I'll give it a try when I get a chance, thanks very much for working on this (and I'm sure you'll also make the losas very happy) [17:07] jelmer: that's only conflicts in NEWS, ignore them, news_merge is not configured on lp :-P [17:07] jelmer: I only quickly skimmed config.py in bzr-svn, there seem to some duplication that may need to be removed though [17:08] s/to some/to be some/ [17:09] jelmer: Overall: my feeling is that we were *very* unlucky to encounter concurrent writers, my proposal use the transport API to make the config write atomic, so this fixes the bug, [17:10] s/,$/./ [17:12] jam: what was your tweak for https://code.edge.launchpad.net/~vila/bzr/cleanup/+merge/28306 ? osutils lazy imports ? Or was there more ? [17:12] vila: there are a lot of imports running on the import slaves I guess [17:13] jam: nevermind, I was looking at the wrong mp [17:13] vila: I think just the osutils stuff [17:13] jam: yes, sorry [17:25] I've accidentally added my email to my name in all my commits, so I'm seeing something like: 'Josh Brown josh@host.com ' - is there any way I can edit the logs to remove the duplicated email address? [17:32] I am confused. Why would I be getting a Contents conflict when I cherrypick a revision that adds a file? [17:33] oh, because I don't understand what -r N.. does [17:50] Eeek. This scares me. [17:51] I have a bzr-svn branch, and did a merge, and got bogus merge results [17:51] I ran the repository through fast-export|fast-import, and got saner merge results [17:51] bzr-svn must be doing something evil [17:56] maxb: next time, try 'bzr st --show-ids' [17:56] vila: ok, what am I looking for? [17:57] maxb: content conflicts are generally about conflicting *paths* with different file-ids [17:57] ah, right, you are speaking regarding my first problem [17:57] That turned out to be me misunderstanding the exclusiveness of the N in -r N.. [17:58] ha, sry for the noise then [17:58] np, you are right about the issue - I was merging a modify on a file-id which didn't have its addition merged [17:58] the other issue is unfortunately rather scarier [18:19] vila: I realize you are probably gone, but any thoughts on bug #530584 [18:19] Launchpad bug 530584 in Ubuntu Distributed Development "File conflict when merge back packaging branch to upstream (affected: 1, heat: 6)" [Medium,Triaged] https://launchpad.net/bugs/530584 [18:19] like, does your resolve stuff have a 'bzr resolve --mine" for deleted files? [18:20] jam: almost done but not quite, --mine should work for deleted files as well as long as they were deleted by the merge [18:20] AFAIR [18:20] vila: the point is to leave it deleted [18:21] supersede changes in favor of considering the file deleted [18:21] I don't fully understand what is flowing where yet, so i'm not 100% sure what is OTHER and what is THIS [18:21] ha, yes, just -rereading the bug report [18:22] that reminds me of some thoughts I had about making some --mine/--other "sticky" but I don't have the context anymore :-/ [18:23] something like a merge plugin based on a list of (file-id, --mine) or something [18:24] or file path... anyway, some text file a user can still modify [18:27] vila: have you upgraded your VBox yet? [18:28] jam: no, given the changelog I see no urgency [18:28] vila: oh, and I was timing 'ping bazaar.lp.net' and found it at 120ms from babune, is that normal? [18:29] I thought you had 20ms ping time to lp [18:29] jam: ha, right, I forgot to comment on that, no 20ms is indeed the "normal" lag [18:29] at what time did you try ? [18:29] just tested again and seem 120ms [18:29] right now [18:29] wow, indeed, never saw that ! [18:30] +20% packet loss [18:30] but maybe because I used ^C [18:30] 6% packet loss here ^C too [18:30] yeah, without ^C and just doing -c 10 no loss [18:30] after your mail I tested again and I got 20ms as expected [18:30] I think it is just 1/num succeeded [18:30] so it may depend on time of day ? [18:31] absolutely possible, but surprising nonetheless [18:31] that is a rather large swing [18:31] and you still get *way* better downloads from http://bazaar.lp.net than I do [18:32] at this instant, I have 110ms ping time (so lower than you) [18:32] and it 'bounces' trying to cap 3Mbs [18:32] although it does a better job than it did the other day [18:33] and it just dropped to 200kB/s for a second or to [18:33] 2 [18:33] hmm, the osutils tweak is more tricky than I thought, both 'import tempfile' and 'from tempfile import mkdtemp' are *required* [18:35] of course I didn't ran the tests after such a small tweak :) [18:40] vila: that's why we have pqm. Because I've certainly done things similarly. [18:40] :-D [18:42] vila: for your config locking patch [18:42] should we be putting 'held' in the directory directly [18:42] or should we be using a 'lock' subdir [18:42] and what is the overhead time of locking/unlocking repeatedly? (how long are the locks maintained, etc) [18:43] the locks are maintained very briefly we just output the config file textual content [18:44] vila: sure, but I just did some timing and "lock_write() && unlock()" on Windows takes 10ms [18:45] I think people won't mess up with a directory named 'lock', I'm less sure of that with a 'held' file and/or 'info' [18:46] vila: also, what is the concurrency that we are trying to solve. Does it make sure to hold the lock open from the time the file is read until it is written? [18:47] jam: No ! Only around the file being written. How could I forgot to explain that :-( [18:47] could I have forgotten ? [18:47] vila: I thought the problem with svn concurrency was that the read got out of sync with the other one [18:47] was it actually having 2 writers on the same file? [18:48] AIUI yes, hmm, yeah, that was mentioned on IRC I think [18:48] and since the fix use an atomic write, the risk is really very low to mess up a reader [18:49] It would require interrupting a read() of an opened file [18:49] vila: if the problem is not using atomic write, then just doing that would 'fix' it, without the overhead of a lock dir [18:49] and needing stuff like break-lock [18:50] no [18:50] the sequence is: lock; read ; set new value ; write; unlock [18:51] no lock means races all over the place [18:51] vila: you just said "No ! Only around the file being written." [18:51] and even harder to diagnose bugs [18:52] you then said: (12:50:16 PM) vila: the sequence is: lock; read ; set new value ; write; unlock [18:52] eerk [18:52] so which is it? [18:52] dang it, I changed it after discussing it with lifeless this morning [18:53] s/read/re-read/ would be more appropriate, but I've changed the method where the @needs_write_lock is used [19:02] jam: fixed, I have to EOD, I'll try to provide a better explanation tomorrow [19:02] jam: just pushing for now [19:04] jam: and to answer your question: *it* is: lock ; re-read the config file ; set the new value ; write the config file ; unlock [19:04] vila: k [19:04] have a good night [19:04] and @needs_write_lock must be set on all methods that *calls* _write_config_file [19:05] and I have a bug in my test for catching this regression [19:13] s/for catching/for not catching/, typo's day until the end... :) === FryGuy_ is now known as FryGuy- [19:50] hi, can I ask a bzr-svn question here? [20:14] yes, I probably can't answer, but ask away, someone can help. [20:14] dipnlik: ^^ [20:18] oh, nevermind, now it's working again [20:18] good to hear [20:18] for future reference all things bzr can be asked here. [20:18] some know more than others, but all questions get answered eventually [20:19] well, maybe not all. but just about all. [20:19] i had a problem with tags, i created a tag with bzr but bzr-svn didn't auto-create the equivalent svn tag [20:19] ah [20:19] just curious how did you get it working? manually create the svn tag? [20:19] I ask because I'm eventually going to be lead person on migrating from svn to bzr [20:20] after i created the first tag manually, the second one worked as expected [20:20] "manually" == "using svn cp" [20:21] and here we don't really intend to migrate the svn server, I just use bzr as a client because it's more user friendly and for some cheap local branches [20:22] sad thing is that when I merge local branches to the trunk their log messages don't go the svn repo, so I don't use them as much as I wanted to :| [20:24] right, that's why we eventually will get rid of svn completely [20:24] my plan is to get them to use bzr as a client first [20:24] get used to the client, then switch out the server, [20:25] oh, nice plan [20:28] rubbs: didi you read http://www.szakmeister.net/blog/2010/may/30/bzr-svn-round-2/ (and the first one, linked on the 1st paragraph)? [20:29] I didn't read it all but it looks interesting [20:32] no I haven't, but i will book mark it thanks! [20:32] bookmark* [20:55] If I have a bzr repository that I want to make public, but it contains some confidential information that I want to keep private, what should I do? [21:12] exarkun: you might want to look at 'filter-branch' from bzr-fastimport [21:12] it should allow you to rewrite the history into something similar, but with some bits removed [21:12] Cool, thanks [21:41] morning [21:53] morning lifeless [21:54] hiya [22:11] vila: [22:11] + __doc__ = """\ [22:11] + Break a dead lock on a repository, branch, working directory or config file. [22:11] I'd prefer to see that spelt as [22:11] __doc__ = \ [22:11] """Break... [22:12] because having to actually think about what you were escaping hurt my brain [22:16] lifeless: that's your brain telling you that you missed what is wrong (in both cases the newline is escaped), and what is wrong are the spaces I've added :-P [22:16] * vila lower the volume and goes back to bed :) [22:16] ...don't actually *need* to escape that docstring [22:16] *newline [22:17] it gets pulled off before anyone tries to print it [22:17] vila: in one, the \ is inside the docstring content [22:17] vila: so you [22:17] 're escaping ^M [22:18] mgz: no, lifeless got it right, there should be no spaces before "Break" [22:18] and then the formatter strips it [22:19] the newline/spaces before 'Break' will always get stripped though, no? [22:19] and in the other I escape ^M too and the scanner strips it [22:19] mgz: no [22:19] some people do print blah.__doc__ rather than help(blah), but they're silly. [22:20] * vila off [22:20] okay, night. === nlisgo_ is now known as nlisgo [22:21] I'm still confused as to what doesn't follow pep 257 stripping rules though. [22:22] the whole point of them is so you don't need to do weird backslash or dedent things. [22:29] why would you ever type '__doc__ = ...' instead of inlining the triple-quoted string? That's just noise [22:30] TresEquis: -O0 [22:30] TresEquis: there are two sorts of docstrings in bzrlib [22:30] TresEquis: programmer docs and command help [22:30] TresEquis: we don't want stripped binaries to lose the command help. [22:31] mgz: I know, thus my asking vila to change ;) [22:31] scripts don't need stripping, anyway [22:31] TresEquis: commands are not scripts [22:31] they don't even get compiled [22:32] TresEquis: you've added an untrue assumption to the discussion [22:32] pydoc bzrlib.builtins [22:32] 'bzr' is the script [22:32] and it has no functions or docstrings or anything [22:32] yeah, it's not like git with lots of teeny shell scripts [22:32] ...is git even like that any more? [22:32] you don't need to use -OO ever [22:32] mgz: fsvo of [22:33] TresEquis: the windows packagers folk saved a MB or something doing it [22:33] TresEquis: so we happily acquiesced and helped them do it. [22:33] it was me. [22:33] easy change, was going for a slight startup speedup [22:33] docstrings aren't the same as comments [22:33] mgz: thats right [22:34] but whatever [22:34] TresEquis: I don't know why comments come into this [22:34] and I think having special, user-visible docstrings marked differently in the source is a good thing [22:34] sounds like you are using docstrings where you could use comments [22:34] TresEquis: the compiled python files *are* smaller without docstrings for the bulk of the library. [22:34] even if it's not as significant a win as it might have been [22:34] TresEquis: not at all. The bzrlib API has docstrings describing how to use functions in it. [22:35] if they need to be runtime-introspectable, then they shouldn't be stripped [22:35] they don't. [22:35] if they don't need to be runtime-introspecctible, then they could just be comments [22:35] TresEquis: that would make pydoc useless [22:35] there are two different cases here [22:35] TresEquis: I really don't know what you're saying here. [22:35] bzrlib being good to develop on [22:36] and bzr being used as a tool [22:36] we want decent docstrings for the first [22:36] if they need to show up in pydoc, then they need to be runtime introspectible [22:36] TresEquis: not from a binary .exe [22:36] but no one with a packaged bzr cares about the method descriptions [22:36] TresEquis: there are two separate sets of docstrings [22:37] nolo contendere, I guess [22:38] there's a thing about "never use -OO" in the python world, but this is one of the few sensible cases [22:39] * exarkun disbelieves [22:39] which bit? :) [22:40] docstrings are great [22:40] Python should try sucking less. There's no particularly good reason for docstrings to make things bloated or slow. [22:40] right, so there shouldn't be a disincentive to use them over comment when you're shipping pyo files only [22:41] *comments [22:41] it's not uncommon with py2exe I think. [22:42] py2exe is a different case, isn't it? [22:42] it's what bzr uses for the windows package [22:42] It gives you an opaque blob you can ask Windows to execute. Even if there are docstrings in there, you can't really find them, right? [22:43] right, so, why not remove them, which is what the change is. [22:43] oh, I see [22:43] I missed the one comment above where someone said "windows" ;P [22:44] as far as I can see, not an approach that works on nix, as users and developers will be working from the same package [22:45] so they'll always be on pyc anyway [22:45] mgz: debian/ubuntu can build pyc's too - its an /etc config option [22:45] mgz: whats tricky is choosing which level of pyo to build [22:45] mgz: s/pyc/pyo/ [22:46] yeah, and that's not something barry's __pycache__ thing has done anything about I believe [22:46] trading off the cost of this discussion vs. the bandwidth for the downloads ;) [22:46] TresEquis: :P [22:47] the actual cost is likely to be in confused plugins devs. Need to do a post to list now there's a windows packaged 2.2b3 (there is, right?) saying what changed [22:49] lifeless: you didn't push a testtools fix for u""? [22:49] mgz: let me check [22:50] pushing now [22:50] sorry [22:51] 's okay, just needing it now. [22:51] ok, someone else pushed, pulling that in first [22:51] hm, and no, no packaged 2.2b3 for windows yet it seems, I may as well draft the message now though [22:52] I'm not seeing anything on launchpad testtools trunk after your rev [22:52] did you uncommit locally? [22:52] mgz: actually no, I did push it. [22:52] mgz: I just have the edited version locally [22:52] rev 75 [22:53] revno: 75 [22:53] committer: Robert Collins [22:53] branch nick: trunk [22:53] timestamp: Thu 2010-06-24 20:03:30 +1000 [22:53] message: [22:53] Do not generate str objects for StringException's internally. [22:53] its just not py3 happy [22:53] yup, I see that. [22:53] you were going to add _() [22:54] I did in fact [22:54] but I hadn't committed it [22:54] and then forgot and ... [22:57] thanks! === beuno_ is now known as beuno [23:17] hello all [23:17] hi jam [23:47] hi poolie [23:47] calling now [23:47] hi there jam [23:48] ok