[01:09] hi spivvo? [01:21] Hi poolie! [01:21] * spiv waits for the cold&flu pills to kick in === idnaria is now known as idnar [03:02] hullo there [03:02] oh what a shame [03:25] spiv are you well enough to talk over the gc cycle thing with me [03:26] poolie: probably [03:26] here or phone? [03:27] Here's probably better given the state of my sinuses [03:27] yeah [03:27] I might make the odd unpleasant sound :) [03:27] :) [03:27] so basically, vila implies that we ought to manually use weakrefs between application type objects [03:27] even when we don't want actual weakref type behaviour of "i don't care if it's there or not" [03:27] I do feel like a missed a memo about that somewhere [03:27] me too [03:28] hm [03:28] I don't really know what the justification is, exactly [03:28] so, for example [03:28] if python was not capable _at all_ of dealing with cycles [03:29] we would need a strict design rule about avoiding them, either by this method or something else [03:29] (and i'd mostly prefer the "something else") [03:29] but... is it really that bad? [03:29] I'm sure it probably helps memory be reclaimed sooner, but I'd really like to see some quantification of how much and how soon. [03:30] I guess my big question is: why do we (or at least vila) believe avoiding these cycles is significant benefit? [03:31] yes [03:31] I can certainly imagine it *might* be [03:31] and, secondly [03:31] if that problem exists, are weakrefs the best solution to it? [03:31] And I could also perhaps be sold on an argument that this forces a bit more thoughtfulness about object lifetimes [03:32] Hmm [03:32] i think a design rule of "be thoughtful about object lifetimes" makes sense [03:32] Weakrefs are certainly the most obvious solution :) [03:32] hm [03:32] i think the most obvious solution is to refactor the objects so there are no cycles [03:32] sometimes this is hard [03:33] But I suppose there are other options like splitting apart the objects more — right. [03:33] but, here, having both the branch and the config each point to a lock would do [03:33] and it doesn't seem that hard [03:33] (modulo foreign format and api versioning stuff) [03:34] I'm also not sure if the benefit is meant to be for CPython or some other VM [03:34] right, obviously this will vary by vm quite a lot [03:34] Right. [03:36] (And it wouldn't totally shock me to find that weakrefs cause surprising differences in GC behaviour on some VMs) [03:36] right, or that they make things much slower [03:37] or more precisely access through them may be slower [03:38] So I'm quite looking forward to the mail you asked vila to send, in the hope it answers some of these questions. [03:40] i might send one first [03:40] and he can respond [03:40] More mail == more good! At least in small doses :) [03:44] lifeless: does that ring any bells with you (https://lists.ubuntu.com/archives/bazaar/2011q2/072972.html) ? [03:47] mmm [03:48] so I think some python implementations are worse than others [03:48] would you like me to reply to the list ? [03:48] probably yes [03:49] s//yes, unless you'd really rather prefer irc [03:49] list is fine [03:49] you pinged me is all :) [03:57] this ws doc is very good :) [03:58] ? [04:00] your service analysis proposal [04:00] ah, cool. [04:00] thanks! [04:33] poolie: replied to that gc mail [04:33] thanks [05:24] Hmm, what version of testtools does PQM have? [05:44] Hopefully whatever is codified in bzr-landing-dependencies [05:51] http://package-import.ubuntu.com/status/php5.html#2011-05-30%2002:58:42.080261 feels potentially transient - can I get a requeue to check? === maxb changed the topic of #bzr to: Bazaar version control | try https://answers.launchpad.net/bzr for more help | http://irclogs.ubuntu.com/ | Patch pilot: jam | UDD failure ratchet: 487 [05:53] maxb: hmm, I suspect it's not, but sure. [05:54] eglibc also has the same issue [05:54] oh [05:54] Whoops, I thought this was a unique new one [05:55] Oh well, requeued anyway, just in case :) [05:55] I think this is a failure revealed by the "remove stale connections from possible_transports" fix [06:51] hi maxb [06:51] ooh nice topic [06:51] :-) [06:51] So, somehow, the UDD importer has sprouted another test failure on natty since the sprint [06:52] It doesn't feel particularly complex, but I've no idea what's changed to cause it to happen now but not then [06:55] hm [06:57] * maxb wonders why BzrDir.sprout takes a source_branch argument, and then ignores it [06:59] maxb: it doesn't? [06:59] It certainly seems to in my vim :-) [06:59] Or with less layers of negation: it isn't ignored, it's passed to _find_source_repo [07:00] (in trunk; the code was a little different in 2.3, but still not ignoring source_branch IIRC) [07:03] whoops. too many matches in close proximity, I didn't spot that one [07:14] ok, no more mail for today [07:31] maxb: yep, php5 still fails the same way [07:32] hm [07:32] I have it running locally, and all it has accomplished so far is to make my laptop quite toasty warm to the touch :-) [07:33] maxb: I suspect the possible_transports includes a smart transport that is still in use, or something like that [07:34] Or that a transport that's not safe to re-use there, e.g. you can't pull from a repo (get_stream) and push back to a repo (insert_stream) on the same smart connection at the same time. [07:35] It's interesting to me that it's a condition that's hit so rarely though. [07:35] hm. I wish we had a command "bzr delete-tags-not-in-ancestry" [07:37] maxb: should be easy enough to write that plugin... [07:38] yes. It's more of a case of "Nooooo! Not another mental context switch!" :-) [07:39] As always, it'd be good to know what the pre-existing request is in the TooManyConcurrentRequests traceback [07:39] Hmm, [07:40] I filed a bug about something that would address that a few weeks ago [07:40] https://bugs.launchpad.net/bzr/+bug/745477 [07:40] Ubuntu bug 745477 in Bazaar "Keep a record of recent events from bzrlib subsystems to use in crash reporting" [Medium,Confirmed] [07:47] :) [08:02] hello guys, how are your cycles today ? [08:02] * vila ducks [08:02] kidding, poolie: I answered your email [08:03] in summary: let's not start a jihad against gc cycles, this has never been my intent [08:09] spiv: /me blinks, the failure for your start_bzr_subprocess log details was related to testools ?!?! :-( [08:21] vila, what do you mean by [08:22] " Such discussions tell me that this [08:22] > doesn't work. That is far more concerning that using weakrefs or not to [08:22] > me." [08:22] That trying to do minimal implementations to make reviews easier tend to trigger issues unrelated to the proposal slowing down the whole landing [08:23] There should be a better way to deal with that [08:23] hm [08:23] Either there is an obvious better solution to use and the proposal and that should be done before landing [08:24] Or the issue should just be discussed and a new proposal will address it [08:25] hm [08:25] i agree in general [08:25] I ran into the case (as a reviewer) with i18n : I can't see an obvious way to address an issue and I tell Naoki: sorry, you can't land this [08:25] I feel really bad saying that :-( [08:25] using weakrefs where they're not obviously necessary doesn't seem like the minimal implementation [08:25] this is one case where we can probably land things and continue the conversation [08:26] as i think you have [08:26] right, but what do you propose as an alternative, I'm fine doing a followup for that [08:27] just tell me "I'd prefer this" not "I don't like that" [08:27] do you know whether weakrefs increment the referents refcount? [08:27] the former leaves me with a way to go while the later is just blocking me [08:27] i guess no [08:27] ('you' and 'me' being general, not specific) [08:28] well, i think many of your things it's just been "I don't understand this" [08:28] so, since the proposal, I had several discussions and I'm not sure anymore weakref is as simple as I thought [08:29] :) [08:29] I still think it's less costly than leaving python handle it but... meh, I can live with just a comment saying, hey look, we create a cycle there, gnark gnark, go python have fun [08:30] why do you think that? [08:30] think what ? That it's harder for python to break this cycle than to not have to worry about it ? [08:31] AIUI python gc will break it, it just requires more work [08:31] so 'have fun' == 'go, try harder' [08:31] i can believe it takes more cycles [08:31] is it more, or less, than dealing with the weakref? [08:32] my gut feeling is that it will be very surprising that it takes less [08:33] the weakref "just" dereferences once [08:33] and only for lock/unlock which doesn't occur often either [08:33] don't [08:34] let's try again [08:34] ok, so it's basically just gut feel [08:34] For me, here, using a weakref is as prophylactic as using free(ptr) ; ptr = None [08:35] hm [08:35] so I know for sure I won't try to call free again on ptr (free implementations that barf on ptr == NULL (gha NULL not None) should just die) [08:35] sure [08:36] though, today, in C, with valgrind, that code is no longer such a good idea [08:36] or not unambiguously good [08:36] but anyhow [08:36] I can change my habit and forget about it [08:36] mm [08:36] It would make a bit more happy to understand what problems *you* have with weakref here though... [08:36] i think having one person adding weakrefs out of habit would be bad [08:37] also, relying just on gut feel as to what ought to be faster is pretty unreliable [08:37] hmm [08:37] And what is your gut feel here ? [08:38] i am trying to avoid making assumptions about how python performs [08:38] i think the reasonable base assumption is that it works as documented, ie unreachable objects will be gc'd [08:39] if we find a particular case where it's slow, let's work out how to fix it [08:39] if we can generalize from evidence that a particular pattern ought to be avoided, great [08:39] i don't think we have that evidence yet so let's just write the simplest thing [08:39] haaaa, that I can buy easily, so what you're proposing is to just use self.branch and be done with it ! [08:40] yes, of course [08:40] sorry if that wasn't clear [08:40] gee, no it wasn't [08:40] but s/wasn't clear/vila didn't get it/ :) [08:40] vila: I replied to your mail [08:41] vila: to add to the conversation in here, I wouldn't assume that weakrefs are necessarily less work for the GC implementation than a regular reference. They're certainly *different* work. [08:43] vila: and yes, +1 to just ignoring cycles and letting Python take care of garbage collecting them. [08:43] right, so basically we don't know, I'd be happy to write tests so we all get a better understanding but I believe this is not a priority [08:43] Right [08:43] no, it's really not [08:43] s/believe/damn well know/ [08:43] If we had infinite spare time I'd be curious to know how CPython copes with e.g. 10000s of weakrefs [08:44] And of course there's the runtime cost in our code of resolving the weakref.ref instance to the actual object [08:45] yup, I start to realize weakref *implementation* doesn't match my mental model which roughly was more along the lines of: takes a ref but don't increment the refcount [08:45] Right, the implementation has to be more complex than that [08:45] so [08:45] fewer lines of code, faster reviews, generally [08:45] So that if e.g. that object is deallocated and a new one allocated at the same memory location the weakref doesn't accidentally hand you the new object [08:46] vila: it may also interest you to know that it took a few releases of CPython before they really worked out all the bugs in how weakrefs interacted with the gc [08:47] vila: there's some interesting comments in gcmodule.c and a .txt file near it in the source with some of the gory details [08:49] so are we all agreed? [08:54] The default is now urllib2 isn't it? Not curl? [08:55] morning all [08:55] The reason I'm asking is essentially about SOCKS proxy support, in both bzr and ubuntuone-client [08:56] hi jam [08:56] ubuntuone-client at least uses python-libproxy, which means it can cope with a PAC script, but then does not know what to do about SOCKS proxies. Bazaar does not use python-libproxy, and presumable doesn't know what to do about PAC scripts. [08:57] hm [08:57] requires running arbitrary javascript iirc? [08:58] Both of them are high-profile Canonical projects, written in Python, so it would seem that they could both benefit from coping with SOCKS and doing it the same way... the personal itch I'm scratching is that I have a PAC script I use to switch proxies at work / home, but it doesn't really work very well with many things that don't support SOCKS [08:58] without checking, i think we do use urllib2 by default [08:58] poolie, libproxy provides the PAC script runner - ubuntuone-client uses the python binding of this library [08:58] patches to use libproxy would be welcome [08:59] poolie, I looked at ways of handling socks with urllib2 and they involve monkeypatching it's socket implementation, I think [09:00] :/ probably [09:00] how does u1 do it? [09:00] Not to say I want to see a switch to curl - it was far more hassle AFAIR [09:00] jelmer: hi? [09:00] U1 doesn't - it only copes with HTTP proxies [09:01] poolie, spiv, vila, jelmer: time for standup? [09:01] yes [09:01] spiv gave his apologies [09:01] also, jr, hopefully [09:01] but jelmer and jr seem to be asleep [09:01] morning [09:01] Riddell: ^^ [09:01] ... or, to have adifferent nick to what i thought :) [09:01] hello [09:02] I'm not asleep, I'm the only one in the mumble chatroom [09:02] touché [09:02] hehe [09:06] I guess you boys will be back in 20 mins or so if you are having a stand-up [09:06] * awilkins contemplates monkeypatching sockets [09:21] awilkins: I know there was a python-dev discussion about how hard it is to override the socket interface for the http libs [09:23] jam, A common answer seems to be "PySocksify", which involves patching the whole "socket" implementation [09:25] http://stackoverflow.com/questions/2317849/how-can-i-use-a-socks-4-5-proxy-with-urllib2 [09:26] Does setting socket.socket affect everything, or just the module-local import of socket? [09:27] global [09:27] I'd suggest just using a LD_PRELOAD socksifier [09:27] yes [09:27] tsocks or socksify [09:27] tsocks is what I use [09:28] it has the very useful feature of conditionally socksifying based on destination address [09:38] I use tsocks also, but it's awkward to use it on things that start up with your session. [09:39] Like ubuntuone-client [09:52] jam: oh, i guess in particular, reviewing udd mps too would be good [09:52] i have some alpha code that gives you a tun device that redirects into socks [09:52] i wonder where [09:53] jelmer: looms push don't know about lossdy, 1 failure (well 3 times the same really) http://paste.ubuntu.com/615680/ [09:53] https://code.launchpad.net/~mbp/+junk/tunnel-forward [09:54] vila: I'll have a look - thanks [09:55] jelmer: great, thanks to you, the hidden question was: will he look or should I ? :D Not urgent === hunger_ is now known as hunger [09:56] vila: :) [09:59] all: false alarm regarding pure python compat, it was just out-of-date extensions [10:00] maybe in july we should try the sip conference system [10:04] That might be nice; mumble is still flaky here [10:04] I'll be sure to boot into natty rather than oneiric for next weeks' standup [10:11] wow, oneiric already? [10:11] is it good? [10:13] It's pretty good, a lot of it is already GNOME3, and it's fairly stable usually [10:21] ok; good night [10:22] have a good evening, talk to you later [10:39] Riddell: I'm back around whenever you are [10:39] jelmer: Do you have SIP set up already? [10:39] we could try it (I have it set up here, and I know poolie does) [10:40] I've not used sip [10:41] Riddell: I found it reasonably easy to set up ekiga, as we have instructions on the canonical wiki [10:41] I used it for the sprint [10:41] since paying for 8hrs of phone time would have been bad :) [10:43] jam: I don't have it set up yet, but I might as well have a look now if we're going to use it later [10:43] jelmer: any chance you can kill jelmer_ ? [10:43] anyway [10:43] jelmer: I think poolie would like to switch, but I don't know any time frame [10:43] mumble tends to work better for large groups (IME) [10:43] but still has stuff like poor echo cancelation [10:44] actually, Skype still has the best echo cancelation of any programs I've used [10:44] Skype has worked best for me too, both on Ubuntu and Android [10:45] I find Mumble works reasonably well compared to most SIP apps on Ubuntu, but the Android app is still not quite there yet [10:46] ah, I need to ask sysadmin for a SIP account first [10:47] jam: well shall we chat on mumble? [10:48] Riddell: mumble's fine for me [10:48] and yes, you have to email rt@ to get your voip account [10:49] jam: you can't hear me? [10:49] Riddell: I can't hear you now, it isn't turning your lips read [10:49] red [10:49] so it isn't getting your "activation" volume, or whatever [10:50] jam: still no? [10:50] the icon goes red here [10:51] still nothing here [10:53] Riddell: I still don't see your lips go red [10:53] I can try relogging if it is maybe on my side. but you can hear me, right? [10:53] jam: no I can't hear you [10:53] relogging now [11:57] Riddell: can you still hear me? [11:57] jam: nope [11:57] Riddell: ok, I'll reconnect [11:57] I can hear you [12:02] Riddell: Did I go silent again? [12:02] yes [12:02] :'( [12:03] Riddell: do you have another voice program we can try? [12:03] jam: could try skype [12:05] Riddell: see private message [12:11] jam, vila, Riddell, poolie, spiv: UDD meeting in #ubuntu-meeting for those interested [12:11] jelmer: are you sure it is this week? I thought this was the off-week [12:12] jam: Yep, at least according to the wiki page [12:15] mgz: when you get this, I think we just need a live chat about: https://code.launchpad.net/~gz/bzr/lazy_hook_test_cleanup_785054/+merge/61586 [12:15] I don't think I can just review the patch [14:36] hi guys. [14:36] I have failed to commit into a folder I have write access to with bzr. [14:37] the folder is owned root:root, and I'm committing as another "regular" user. [14:37] BUT [14:37] There are ACLs, and the user can create files/folders inside repository's folder with UNIX shell. [14:37] Is it possible bzr assumes UNIX permission model and avoids trying operations on files/folders owned/permitted badly by "UNIX" convention ? [14:38] I did dig the bzrlib code., [14:39] sbarcteam: IIRC bzr doesn't check first (in this scenario), it just attempts to create (trusting that the OS will return a permission error if it doesn't have permission, etc). [14:39] sbarcteam: pastebin the error you get? [14:40] the specific operation bzr fails upon is this locking LockDir [14:40] and it gives me some kind of file. [14:40] does lock files creation require special kind of access list entry ?! [14:42] Please, paste the text of the actual error into http://paste.ubuntu.com/. Perhaps run the command with -Derror to force bzr to give a traceback too. [14:42] That way we can tell which specific filesystem operation is failing. [14:43] I'm having trouble fixing a mistaken tag. It seems no matter what I do I get conflicting tags when I try to push the change. Would someone point me to some documentation about how to handle this (I did what I thought bzr help tag was suggesting)? [14:44] ScottK: 'bzr tags -d $source' is presumably listing a different rev for that tag than 'bzr tags -d $dest'? [14:45] ScottK: either reapply the tag delete/rename/update directly to dest too, or just delete it in dest then push. [14:52] spiv: Thanks. [15:31] jam: I'm around briefly, but out again shortly. I should be around all tomorrow though. [15:32] mgz: either way [15:32] spiv: eerk, you're around ! I think I fixed bug #788235 ;) [15:32] Launchpad bug 788235 in Bazaar "blackbox.test_serve.TestBzrServe.test_bzr_serve_port_(readwrite|readonly) hangs on maverick " [High,Confirmed] https://launchpad.net/bugs/788235 [15:32] I'm gone the rest of the wek [15:32] week [15:33] okay, let's go over it quickly now [15:34] vila and I discussed it a little a few evenings ago, my real problem is I don't get how hooks are meant to work now [15:34] they aren't (?) :) [15:34] hehe [15:35] mgz: so is the issue you aren't sure if hook dicts should be completely empty? [15:35] or whether we should be able to reset them, or what? [15:35] I'm not sure what work needs to be done to get the level of hook isolation tests expect [15:36] currently the old code iterates over known hook objects and fiddles with their attributes [15:36] and when lazy hooks were added, some (broken) code for emptying and refilling the new _lazy_hooks dict was added [15:37] but when trying to make that code do what it appeared to be wanting to, I found 1) it made no difference in practice and 2) when I tried to test with a new hook point rather than a known one, isolation didn't work at all [15:38] 2) is because you didn't register your hook (not hook point) [15:38] I call install_lazy_hook / the other install method [15:38] what else needs doing? [15:39] anyway, apart from some ugliness in the tests, what I'm trying to verify should be pretty clear from the code [15:39] the misunderstandings are because we tend to call hooks what the code calls hook points (except for lazy_hook which tends to blurry the lines even more) [15:39] mgz: have you tried registering your hook ? [15:39] I... don't know what that would look like [15:40] I take that as a no then ;) [15:41] something like hooks.known_hooks.register_lazy(your_test_module, your_hook, your_hook_class) [15:42] okay, so how do I isolate *that* to the test... [15:42] hehe [15:43] with a lot of effort ? [15:44] mgz: presumably you'll have 2 levels of TestCase, right? [15:45] the inner one which actually does some trivial thing, asserting it is isolated [15:45] and the outer one as the bzrlib test case [15:45] the bzrlib case can register the hook point [15:45] and the inner one then fiddles with registering a test-spceific callable [15:46] then when the outer one finishes running the inner one, it asserts that everything is cleaned up. [15:46] yup. [15:46] the current tests have an inner case that installs and calls a hook [15:46] a hook point [15:46] mgz: why is it "_clear_hooks" and "_restoreHooks" one of those is misrepresented :) [15:46] but don't install the hook in known_hooks [15:47] installs a hook point and triggers a hook callback [15:47] ...some name cleanup really would help [15:47] mgz: I use the terms HookPoint and callback [15:47] then the outer case triggers the hook again, the theory being that the hook callback should then *not* be called [15:47] I think the code is a bit confused as well. [15:49] anyway, I'd like this isolation code to just be test.overrideAttr(_mod_hooks, "_dict_where_all_hook_callbacks_live", {}) [15:49] but life isn't that simple. [15:52] mgz: so are your tests not failing? or failing too much? [15:52] yup, both. [15:52] the lazy one using the lock hooks doesn't fail [15:53] and the new hook point that doesn't get into known_hooks fails by, [15:53] never triggering the callback for lazy installation [15:54] and always triggering the callback for eager installation (ie, doesn't get cleaned up) [15:54] ...I think I have those two the right way round [15:54] so, "hooks not in known_hooks are broken" is a fine answer for those two [15:55] ...would be nice if the code actually asserted that [15:59] anyway, I think I don't want to rewrite the hooks code, so may just delete the bogus tests and the redundant cleanup and see if anything else breaks [16:04] okay, I'm off. [16:09] mgz: have a good evening [16:33] argh, the new SSD arrived too late :-( The old one died in flames... [16:35] literal flames? [16:35] :-) [16:39] Probably. Somebody sent it too many nasty emails, and it committed suicide. [16:56] am i right that qbzr is more advanced than bzr-gtk and therefore recommended to be used withing explorer? [16:58] i'm familiar with cli using freebsd (and before that linux), but would like to know to recommend right tool for (potential) contributors who are GUI-fans [16:59] gour: yeah, qbzr is definitely more advanced and better integrated into bzr-explorer [17:00] jelmer: thank you === beuno is now known as beuno-lunch [17:33] cool, it's possible to convert branch into colbranch :-) === deryck is now known as deryck[lunch] [17:56] sigh, lots of lossage, all restored except for babune which I will need to restore from a 3 weeks old backup (of course) and some bits I managed to get back from the ashes [17:57] now to turn the bits into files... [18:00] So, that counts as a test failure, right? 8-} [18:00] hehe, thanks, I will end on that, couldn't have dream better ending :) [18:00] bye folks, have fun ! [18:00] Better hope it's transient, and not repeatable :p === sidnei_ is now known as sidnei === beuno-lunch is now known as beuno === deryck[lunch] is now known as deryck === zyga is now known as zyga-afk === cbz_ is now known as cbz === Ursinha is now known as Ursinha-afk === yofel_ is now known as yofel === Ursinha-afk is now known as Ursinha === Ursinha is now known as Ursinha-nom === Ursinha-nom is now known as Ursinha [22:54] how is performance now of the system? [22:55] good [22:56] i remember it used to be not to good lifeless [23:04] EM03: sure, it used to be slowed [23:34] hi all