[00:32] kklimonda, yes, that's correct [00:33] kklimonda, svn-import imports all of history, it's different from e.g. "svn checkout" which downloads just the last revision [00:42] jelmer: hai [00:44] lifeless, hellø [00:45] https://code.edge.launchpad.net/~lifeless/subunit/nopassthrough/+merge/9557 [00:46] lifeless, any reason for the two spaces of indentation on line 50 and 51 in the diff? [00:47] editor fail [00:47] will fix [01:13] lifeless, why does launchpad add "(community)" behind jml's name and mine in the reviewer field? [01:14] I don't know [01:14] there is a bug open about it, that I filed. [01:15] https://bugs.edge.launchpad.net/launchpad-code/+bug/400950 [01:15] Launchpad bug 400950 in launchpad-code "not clear how to tell code reviews that (community) is wrong for a branch" [Low,Triaged] [01:33] lol. re fail :( [01:33] File "/usr/lib/python2.6/sre_compile.py", line 517, in compile [01:33] "sorry, but this version only supports 100 named groups" [01:33] AssertionError: sorry, but this version only supports 100 named groups [01:35] lifeless: what are you trying to do? [01:36] thumper: just test combination [01:36] I suspect a shallow bug in command line aggregation [01:36] I'll look at it later [01:36] jelmer: it seems there is a small bug somewhere in the community identificaiton code ... [01:36] jelmer: we are looking into it [01:37] lifeless: do you know much about StringIO and unicode? [01:37] sadly yes [01:37] should it be avoided? [01:37] thumper, what is community supposed to be though? Would it be used to identify people who don't have voting rights? [01:38] thumper: its ok some of the time, but it doesn't have clearly (or consistently!) defined semantics. [01:38] jelmer: it is more to indicate that while they may provide a review, it doesn't yet hold the weight of a real reviewer [01:38] lifeless: it should be well defined [01:38] thumper: if you want a bytestream, which is what python 2.x files are meant to be, its a bad bad bad idea to write unicode to the input side [01:38] I know what it should be [01:38] thumper: its not; StringIO and cStringIO, which are meant to be identical, behave differently. [01:39] ew [01:39] thumper: I'm sure it seems well defined though :) [01:39] thumper: (c)StringIO is meant to emulate a file, which is meant to contain bytes. So, feed it bytes (str) not unicode, or else unexpected crap happens. [01:39] spiv: ok, ta [01:39] thumper: in py3 there are defined semantics for layering strings down to bytes and back [01:39] spiv: I'll reject my reviewers suggestion then :) [01:40] Use CodecReader/Writer (or whatever they are called) wrapped around a StringIO, if you like. [01:40] thumper: what did they suggest? [01:40] I'm using text.splitlines()[:max_lines] -> testlines = islice(StringIO(text), max_lines) [01:41] hello thumper, lifeless [01:41] hi poolie [01:41] hi poolie [01:41] thumper: any chance launchpad could stop flattening whitespace in code review comments? :) [01:41] what? [01:42] poolie: ref plz [01:42] thumper: uhm, islice(text.splitlines(), maxlines) [01:42] thumper: is ~= [01:42] also [01:42] poolie: someone must have changed it [01:42] thumper: yeah, that seems pretty dubious to me. splitlines really could use a maxsplit argument, like split. [01:42] thumper: also [01:42] lifeless: I assume the point is to avoid a second copy of the (large?) string in memory [01:43] islice(bzrlib.osutils.chunks_to_lines([text]), maxlines) [01:43] thumper:look at eg john's comments in https://code.edge.launchpad.net/~mkbosmans/bzr/topo_sort/+merge/9532 [01:43] thumper: or just use split('\n', max_lines) [01:43] it's like it was specifically designed by someone who hates python [01:43] ^ that is about the most efficient, if you need to deal with large strings/are doing this a lot [01:43] spiv: that doesn't discard the rest though does it? [01:43] the problem occurs when you copy and paste from the review diff [01:43] * lifeless repeats [01:43] thumper: yeah, I guess not. [01:43] islice(bzrlib.osutils.chunks_to_lines([text]), maxlines) <- use that [01:44] poolie: they don't look flattened to me... [01:44] thumper: its python, your text isn't getting freed anyway [01:44] oh, the other thing you could do is use a buffer object [01:44] but really, at that point, I'd rather write C [01:44] lifeless: what do you mean the text isn't getting freed? [01:45] lifeless: right, scan ahead for the nth \n, make a buffer object, splitlines that. [01:45] text.splitlines() is your big slab of text, its going to be kept in memory until you're done with the last handle to it [01:45] spiv: not quite, N buffer objects [01:45] splitlines will allocate [01:46] depends on parameters thumper hasn't shared about object lifetime [01:46] lifeless: depends on you are worried about, I guess [01:46] ^ great minds [01:46] lifeless: I'm thinking of say 100M of text after max_lines that you don't care about (and certainly don't want a 2nd in-memory copy of) [01:46] I don't care that much I guess [01:46] at most will have 512k [01:47] thumper: I see poolie's problem; it's in the code lines John's quoted from the review diff, not the new code he's suggested in the comment. [01:47] spiv: I was thinking of 20K lines and a little noise after :) [01:47] thumper: e.g. the first four lines of that comment [01:47] lifeless: :) [01:47] thumper: can you give us a quick english description of the problem [01:47] gotcha [01:48] it is actually the tal formatter that formats the diff [01:48] I get a (maybe unicode) string [01:48] lifeless: Well, I'm guessing from the "max_lines" value that limiting the amount of data handled is the concern. It would be good not to guess :) [01:48] to break up and make pretty [01:48] yes, I want at most a configurable number of lines [01:48] ok [01:49] so, here's what I'd do [01:49] I'd first pass the string through some coerce_to_unicode thing [01:49] noting that diffs from bzr can be in any encoding [01:49] lifeless: like unicode(text) [01:49] and that different files can be different [01:49] thumper: like that, but that will fail. [01:50] thumper: there are heuristics things around in lp for this already. [01:50] in exceptional cases, parts of the diff are in two different encodings [01:50] lifeless: we have code that attempts to decode the actual diff elsewhere [01:50] consider when someone reencodes a NEWS file from cp-1280 to utf8 [01:50] (this is real, people do it it Debian.Changelog) [01:50] yes we know there are edge cases [01:50] ok [01:50] thumper: that's just an implicit spelling of text.decode('ascii') (unless you've done evil things to your python runtime, like "import gtk"...) [01:51] well, I'd get it to a known state, either all bytes, or all unicode [01:53] then then loop on re.finditer(r'\r|\n', re.MULTILINE) to get the position [01:53] finally, bzrlib.osutils.chunks_to_lines([buffer(text, 0, pos)]) [01:54] we may need to bugfix that to deal with buffer objects, but its the right conceptual function for stashing that in. [01:54] interesting [01:54] overkill for me now, but interesting [01:54] I thinl its overkill too [01:54] I think diffs will either be fine to deal with naively [01:54] or so big that the first text is already killing you [01:55] so actually limiting needs to be pushed down the stack. [01:56] alternatively, loop on [01:56] re.finditer(r'.*\r|\n', re.MULTILINE) [01:57] for match in islice(re.finditer(r'.*(\r|\n)', re.MULTILINE), max_lines): [02:01] poolie: btw, that case I was worried about turns out to be a non-issue; it's already handled correctly. [02:06] thumper: actually, the re one is worth testing, it may well be the bomb [02:07] :) [02:12] spiv: way to go [02:13] ...and it's beating IDS for wall clock time on fetch from 1.9 -> 2a over HPSS. [02:13] morning [02:14] For bzr.dev -r2000, anyway. It'll be interesting to see how mysql goes. === cody-somerville_ is now known as cody-somerville [02:15] hi Ian [02:15] moin Andrew [02:16] jelmer: hola [02:20] spiv: \o/ [02:21] igc: did you get a chance to take commit for a spin? [02:21] lifeless: not yet - will do so today [02:23] ok [02:28] Hey, anyone here familiar with the FTP transport code in bzr? [02:28] somewhat [02:29] its mainly a thin veneer around ftplib [02:29] if I am remembering correctly [02:29] Someone was having problems with CHMOD not being allowed on their Windows-based FTP server for their host, but from what I read in the code it just warns if CHMOD fails [02:29] perhaps the server is giving a fatal failure? [02:30] Not sure, but I am correct that it should just generate warnings and still push (for example) ? [02:30] It also seems to have problems pushing because it keeps the branch lock open :( [02:31] we catch ftplib.error_perm [02:31] its probably returning a different error code [02:31] the intent is to warn and continue [02:31] Well, the error string for that catch is what's being output [02:31] oh, if its outputting the string then its being caught ok [02:31] I suspect that the FTP is killing the connection at sometime, which would leave the branch locked [02:32] that sounds plausible too [02:32] If that happens, what does bzr do? [02:32] Does it try every X seconds? [02:32] if the branch is locked when we try to lock it, yes we spin for a while [02:32] if the connection is broken leaving a stale lock, the 'bzr break-lock' command can be used to manually break the lock [02:33] Indeed, but his FTP is just too borked I think [02:33] To sync just 1 file it drops the connections so much that you can't realistically break the lock and push, you'd be doing it every few seconds [02:34] ouch [02:34] I need to go eat; perhaps you or he could file a bug? [02:34] we can see if we can improve things - though it may not be feasible [02:34] I believe he is, but honestly I think it's just the FTP being fudge [02:34] Unacceptable for an FTP to not have CHMOD IMO [02:35] I don't know that its chmod causing the problem [02:35] Thanks for your help :) [02:35] it may be incidental [02:35] I agree [02:35] ciao [02:35] Either way his FTP is the real issue here [03:36] vila: I'm going to look at the test suite this afternoon [03:36] I've got my check branch passing again [03:37] poolie: do you want to do an incremental review? The changes were all shallow - tests calling branch.check() rather than check_dwim on the branch path (which is the public API according to our docs) [03:52] lifeless: speaking of reviews, inventory-delta is up for review again. [03:52] lifeless: it uses a separate substream now and is generally looking fairly shiny I think. [03:52] awesomeness [03:52] what did you do about the parents issue [03:54] It transmits them on the FulltextContentFactory. [03:54] cool [03:54] Actually, this is effectively unchanged from the previous interation, I'm pretty sure despite John's comments it was sending them before, too. [03:54] :P [03:55] Anyway, if it's wrong, then a test should fail :P [03:55] But I'm pretty sure this is doing the conservative thing. [03:57] Next up is more interactive testing and maybe see if I can do a cheap hack to do something about the "inserting a stream into 2a causes massive bloat until the final autopack" issue. [03:57] * igc lunch [03:57] But first, lunch. [03:57] igc is a trendsetter :) [03:57] * spiv -> lunch. [05:16] lifeless: hi i'm back [05:16] lifeless: want to read spiv's thing? [05:22] igc, we should chat about the web site sometime, if you're up [05:24] poolie: sure. I'm back from lunch now. I'm feeling rather washed out today but discussing websites is well within my mental ability nevertheless :-) [05:24] igc :) how about now? [05:24] poolie: np [05:25] poolie: wouldn't mind a chat some time [06:53] hi thumper [06:54] omg death by telephone [06:54] mtaylor: ask thumper; here. [06:54] lifeless: i forget, did you want me to reread your check-1 branch? [06:56] thumper: is it possible to use boost::format without using the % operator? [06:56] poolie: its up to you [06:56] 12:36 < lifeless> I've got my check branch passing again [06:56] 12:37 < lifeless> poolie: do you want to do an incremental review? The changes were all shallow - tests calling branch.check() rather than check_dwim on the branch path (which is the public API according to our [06:56] docs) [06:57] that's https://code.edge.launchpad.net/~lifeless/bzr/check-1/+merge/7489 ? [06:58] if you want to use the Trivial Button, use it. [06:59] I'll send it to pqm now then [07:13] lifeless, i'm working up towards doing that subunit-filter feature myself... [07:13] cool [07:14] I've been thinking about the problem of multiple outcomes for tests [07:14] I think its a 2d squashed into 1d problem [07:14] erm, not multiple events per fixture, but addSuccess vs AddFailure vs addSkip etc === davidstrauss_ is now known as davidstrauss [07:14] I might mail TIP about it actually [07:20] meaning it should be more like addResult('success')? [07:21] I think there are multiple dimensions: [07:21] - ran | did not run (reason) [07:21] - failed (details) [07:22] I'm exploring the concept at the moment [07:22] its not mature [07:22] not all coordinates make sense [07:23] didn't run, failed is an interesting one to consider - MissingDependency might be that case [07:23] basically I think the that list can't ever be complete the way its tackle at the moment [07:23] and that provokes ugliness/continual growth in the API [07:23] hi all ! [07:24] hi vila [07:24] agree [07:24] vila: I haven't fixed it yet [07:25] it may be more cleaner to add some kind of object to which you can then add attributes and dimensions [07:25] hi vila [07:25] lifeless: just reading your last comment, you're on on the right design track though :-) [07:26] thanks [07:26] vila: so, for --parallel [07:27] I think special casing the counting decorator to be applied before the parallel one, would be reasonable. [07:27] or something [07:27] long term I want to fix the driving problem about status info [07:27] push that down to subunit and up to python upstream [07:28] first though, its EOD for me, and I'm off to think/tweak nested progress in subunit [07:29] lifeless: ok, short term I just disabled --parallel=fork in the test farm, but 1) it's a shame, 2) I still need it :) [07:29] lifeless: how am i meant to invoke the subunit test suite? [07:30] poolie: make check / make distcheck [07:31] vila: hi [07:32] bialix: hi, wow, already up ? :) [07:33] yep, I'm in +2 TZ [07:33] vila: bzr pull http://bzr.dev hang [07:33] http://pastebin.com/m757ed464 [07:33] bialix: hehe, I know, but you generally come later [07:34] is it bug or this is my network ? [07:34] vila: :-P [07:35] bialix: from what I can see, it should be your network, try with -Dhttp maybe ? [07:35] I've surprised by 25 bytes left message [07:36] our ADSL is so flaky :-/ [07:37] poolie, beuno, igc: incoming email, although I'm outgoing for sleep. :) [07:37] well, it smells a bit but not enough to trigger a deep debug session so far, I've seen it in other reports though so I'm aware that something strange is going on here [07:37] hi emmajane [07:37] emmajane: hi ! [07:37] emmajane: good to see you around :) [07:37] emmajane: sorry I've missed you on irc recently - haven't been around much these last 10 days [07:38] what it means: lock cannot be broken? http://pastebin.com/m3a7d4d30 [07:38] igc, not to mention yo'ure in the wrong time zone for me. :) [07:38] (it's nearly 3AM here) [07:38] emmajane: may be we can catch up some time later this week [07:38] igc, that'd be good! [07:38] bialix: probably something holding the os lock (must die) [07:39] there should be a traceback in bzr.log [07:39] bialix: isn't it OS lock realted (which must die ?) [07:39] rats [07:39] jinx :) [07:39] vila, ps HAI :) [07:39] rats [07:39] ok. sleep time now. [07:39] HAI ? [07:39] lifeless: (lazy) how is subunit meant to treat xfail in its tests? [07:39] sleep well [07:39] vila, "hi" in lolspeak. [07:39] poolie, thanks. :) [07:40] emmajane: :) [07:40] lifeless: nm [07:40] poolie: it doesn't have any xfail tests at the moment :) [07:40] * emmajane clearly needs sleep :) [07:40] night [07:40] poolie: it has tests /of/ xfail [07:40] it folds it to success, it seems? [07:41] emmajane: yep, certainly wrong timezone but I'm often around in my early morning as well so I'm sure we'll catch up soon enough :-) [07:43] poolie: ah yes. [07:43] poolie: 2.7 has an addExpectedFailure now, I haven't adjusted subunit to use that yet, is all. [07:43] vila: -Dhttp does not reveal anything and pull has finished successfully [07:43] but OSLMD definitely [07:44] OSLMD ? [07:44] hi bialix! [07:44] invented today: OS Locks Must Die! [07:44] hi igc [07:44] * vila wonders where he lost his acronym deciphering capabilities... [07:44] OSLMD !!! Of course ! [07:44] LOL [07:45] igc: today will do review of pending merges in qbzr, after noon in my TZ [07:45] lifeless: this seems to mean it loses the KnownFailure exception details :/ [07:46] bialix: excellent. thanks [07:46] poolie: thats a shallow bug now that python has a defined contract [07:46] but today morning I'm thinking about your mail. Why quncommit is so important for you? [07:47] igc: ^ [07:50] poolie: I'm filing a bug on the xfail squash [07:50] i did already [07:50] bug 409193 [07:50] Launchpad bug 409193 in subunit "folding KnownFailure to success loses data" [Undecided,New] https://launchpad.net/bugs/409193 [07:50] oh, thanks [07:51] also bug 409191 [07:51] Launchpad bug 409191 in subunit "want a subunit-sort utility" [Undecided,New] https://launchpad.net/bugs/409191 [07:51] not very related [07:51] but would be cool with many failures [07:51] lp indeed [07:51] bah typo [07:51] indeed [07:51] back shortly [07:56] bialix: it's not to me personally. But it is w.r.t. menu options I'd want to explain in a manual for explorer say [07:58] bialix: scanning over the user guide and 'translating' the tasks to gui actions (instead of bzr commands), we need ... [07:58] I mean: I don't think this is main operation [07:58] ah [07:58] bialix: (1) easy importing & (2) being able to undo mistakes [07:59] easy importing? [07:59] emmajane: really nice mail :) [08:00] bialix: I personally care more about shelve for example, than uncommit, but most new users will screw up a commit message more often than want to shelve ;-) [08:01] shelve (shelve2) does not work for me, so [08:01] bialix: "I have a tree of files - put it under version control for me" [08:01] ok, I've grok your pounti [08:01] ok, I've grok your point [08:01] qimport? [08:02] is not import command still the part of bzrtools>? [08:06] baz-import is, i think [08:06] plain import i think has always been built in? [08:06] import is in core I'm fairly sure [08:07] lifeless: there are no blackbox tests iiuc? [08:07] bzr help import said: From: plugin "bzrtools" [08:07] :-P [08:07] none! [08:08] woo, freedom! [08:11] well that obviously works [08:11] why do people bother writing tests? [08:12] poolie: so, I'll probably add something approximating blackbox tests at some point in the future [08:15] poolie: for now, I do TDD for all the core code [08:15] sure [08:15] and treat UI validation as a shallow release task [08:15] i'm not complaining, i'm just checking i don't miss it [08:15] :) [08:15] i mean i'm changing it and i don't want to submit without tests [08:16] thats fine [08:24] done- https://code.edge.launchpad.net/~mbp/subunit/408821-filter-re/+merge/9680 [08:31] spiv, how's the inventory delta patch? did it turn out it's ok to go? [08:31] ooh this is very nice [08:32] err might not be a string [08:32] it seems like a bug there [08:32] but otherwise nice [08:32] i was only assuming it could be cast to a string [08:33] or i only meant to [08:33] lifeless: I am now [08:33] lifeless: just trying this fast-export thing at the moment [08:38] ok, I've run the export and piped it to a file [08:40] let's see now [08:44] an error - http://paste.ubuntu.com/247816/ [08:48] lifeless: if you have any ideas, give me a shout - I'm off to work now but will pick up hilights. Thanks in advance! [08:55] a bug in fast-export, or a broken stream [08:55] file a bug [09:12] * igc dinner [09:12] vila, lifeless, i think i'll call it a night [09:12] bye [09:12] ciao [09:12] I'm reviewing atm [09:12] poolie: bye [09:13] i wonder how spiv's patch went? [09:19] poolie: reviewed [09:19] the subunit thing? [09:19] I think its nice, but improvable [09:19] yes [09:20] oh, missed a bit [09:20] I don't think the predicate should be public on the filter [09:20] it would be odd for someone else to be calling it [09:21] i thought they might want to chain on it [09:22] you mean replace it after construction? [09:22] yes [09:22] mmmm, I think I'd rather have a list of predicates for that, in the object. [09:23] its also a bit yagni, as I strongly suspect these filters are build-and-execute, not build-edit-execute [09:23] did you comment on the mp? [09:23] yes [09:23] i'll make it private [09:23] bunch of mostly shallow stuff [09:23] it looks really nice as a feature [09:23] oh, now you did [09:25] how can it hit on None? [09:26] perhaps not from the command line [09:27] but addSkip(test, None) is something people might call [09:28] its possible I'm overthinking it [09:31] k, well I'm going to head for a bit [09:31] I'll tackle progress later [12:38] * bialix works on qbzr [12:46] poolie: I resubmitted the inventory-delta patch for review; didn't Launchpad send you mail about it? [12:46] oh hi spiv [12:46] i saw there was a new review from about 5h ago [12:46] i didn't review it myself though :/ [12:47] maybe john will === mrevell is now known as mrevell-lunch [12:55] am I doing something wrong, or is push being strict by default supposed to be super-annoying? :-P [12:59] hehe [12:59] you can set push_strict = False in bazaar.conf to disable this [13:03] ah, nice [13:03] nice? [13:04] it's nice that I can disable that in the preferences [13:04] Tak: when do you encounter it ? [13:04] with qpush you've got yes/no dialog [13:05] vila: sometimes it's perfectly legal situation to push finished job and get unfinished changes in the tree :-P [13:05] vila: when I have uncommitted local changes [13:06] bialix: I know it's perfectly legal, I'd like to better understand why and when people encounter it [13:06] ah [13:06] krisives: ping? [13:06] Tak: but why did you want to push anyway ? [13:06]