igc | morning | 00:22 |
---|---|---|
lifeless | poolie: thumper asked that we not use the queue feature for the moment | 00:24 |
lifeless | hi igc | 00:24 |
igc | hi lifeless | 00:25 |
lifeless | poolie: 'e' for email is the new command in hydrazine | 00:25 |
poolie | yeah | 00:26 |
poolie | you broke my muscle memory a bit there | 00:26 |
poolie | hello igc! | 00:26 |
igc | hi poolie! | 00:26 |
lifeless | poolie: sorry! | 00:27 |
lifeless | poolie: if you pull my hydrazine/cron, it will show you the last comment | 00:27 |
lifeless | so you can see if its been sent to pqm or whatever | 00:27 |
lifeless | ok, first pass reviews done | 00:28 |
lifeless | I'm going to see if I can finish a spike I started in the weekend to make completing proposed things easier. | 00:29 |
lifeless | losa - did something kill stuff on balleny ? I just got an interrupted test run... | 00:44 |
lifeless | spm: ^? | 00:51 |
=== FryGuy_ is now known as FryGuy- | ||
AfC | bzr: ERROR: No such file: u'/home/andrew/vcs/java-gnome/.bzr/repository/obsolete_packs/': [Errno 2] No such file or directory: '/home/andrew/vcs/java-gnome/.bzr/repository/obsolete_packs/' | 03:26 |
AfC | is that normal? | 03:26 |
lifeless | you've deleted that directory. | 03:37 |
lifeless | Don't do that. | 03:37 |
lifeless | and you should recreate it. | 03:37 |
lifeless | jam: thnaks | 04:23 |
AfC | lifeless: maybe bzr could create the directory if it's missing? | 04:28 |
lifeless | AfC: we felt it was better not to | 04:28 |
AfC | lifeless: hm | 04:28 |
lifeless | directories going missing is a pretty nasty event for the filesystem to do to us | 04:28 |
AfC | lifeless: and that error saying the same thing twice is better? | 04:29 |
AfC | lifeless: yes, I can understand that | 04:29 |
AfC | although we've been telling people to "delete the obscelete_packs directory" and if you're saying the nuance is to delete it's *contents* perhaps it's not uncommon to lose the directory? | 04:29 |
GaryvdM | Morning all | 04:33 |
* GaryvdM having a sleep fail :-( | 04:34 | |
lifeless | :< | 04:41 |
GaryvdM | Reading bug reports - sleep fail over :-) | 04:42 |
lifeless | AfC: we've not been telling people to do that. Some folk have said it, I certainly never havwe. | 04:43 |
AfC | lifeless: oh | 04:43 |
AfC | I must have misread what I found in the list archive then | 04:44 |
lifeless | there are people that run pack, for no particularly good reason | 04:45 |
lifeless | except that you had to in git to get performance, at one point | 04:45 |
lifeless | and pack keeps the old files around because some fs's (NFS, ext4) are shonky | 04:45 |
AfC | it must have been from when I had a 5GB .bzr directory for a 20 MB project. | 04:45 |
lifeless | if people want to *immediately* gc, they can delete the contents of that dir | 04:46 |
lifeless | but bzr will gc it after a few commits anyway | 04:46 |
lifeless | (5 on average) | 04:46 |
* igc dinner | 09:56 | |
=== joerg is now known as Guest70592 | ||
kostja_osipov | hello. How do I find out the current format of my bzr repo? | 10:24 |
kostja_osipov | I seem to be unable to find anything in the docs... | 10:24 |
spiv | "bzr info -v" | 10:25 |
kostja_osipov | thanks! | 10:25 |
bialix | heya | 10:28 |
=== bialix_ is now known as bialix | ||
quicksilver | bzr doesn't deal very elegantly with the case where a directory is removed in one branch and some of the files inside are modified in the other | 12:14 |
quicksilver | it deals *correctly* just not very elegantly | 12:15 |
quicksilver | the messages you get don't immediately tell you what's happened. | 12:15 |
lifeless | please file a bug saying that | 12:16 |
lifeless | gnight | 12:16 |
=== oubiwann is now known as oubiwann_ | ||
=== oubiwann_ is now known as oubiwann | ||
=== JFo-swap is now known as JFo | ||
=== Guest70592 is now known as joerg | ||
=== gnomefreak76 is now known as gnomefreak | ||
=== deryck is now known as deryck[lunch] | ||
=== oubiwann is now known as oubiwann_ | ||
=== oubiwann_ is now known as oubiwann | ||
=== deryck[lunch] is now known as deryck | ||
Spyzer | due to interruptions in my internet connection my downloading from bazaar "bzr branch lp:inkscape" is interrupted. How do i resume the downloading from the interrupted point??? | 19:37 |
Spyzer | please help | 19:37 |
Spyzer | ?? | 19:42 |
Spyzer | is there no possible way of what i ask?? | 19:43 |
sebi` | hi, is there a command to blacklist a file (read: preventing a file to appear in the branch)? | 19:44 |
sebi` | since I have a bunch of gen*.sh files, which are mostly useless to the enduser | 19:45 |
=== radoe_ is now known as radoe | ||
Spyzer | how to ab\void downloading history in a bzr branch ?? | 20:06 |
Spyzer | *ab\void avoid | 20:06 |
hno | Spyzer: currently only by having the history locally already... | 20:10 |
hno | been discussions about supporting shallow branches / history horizons, but not sure what the current status of that is. | 20:11 |
Spyzer | hno: if i do a bzr branch lp:inkscape and it interrupts in middle and if later on i use the option --use-existing-dir later on. Shall that resume the donwload the branch form the point it was terminated | 20:11 |
Spyzer | ? | 20:11 |
Spyzer | pls don't mind the typos | 20:12 |
Spyzer | its saying bzr: ERROR: Already a branch: "inkscape" | 20:13 |
hno | Spyzer: No idea, but I guess not. | 20:13 |
Spyzer | how do is\relove this | 20:13 |
Spyzer | oh man, is there no way to resume a download | 20:13 |
Spyzer | ? | 20:13 |
hno | It's only initial branch creation which is this heavy. If you already have branched some branch of the same project before then no need to download much again. | 20:17 |
sebi` | anyone? ): | 20:19 |
Kohlrabi | sebi`: bzr ignore? | 20:20 |
sebi` | Right, thanks. | 20:20 |
mkanat | spm: Did codebrowse get updated to trunk after I mentioned I fixed the fix better, the other day? (A few weeks ago.) | 20:47 |
lifeless | moin | 20:56 |
lifeless | jam: would like to talk network fetch efficiency if you're around | 20:57 |
* hno wonders if bzr wouldn't benefit from a general cache... | 20:58 | |
GaryvdM | vila: ping | 21:00 |
TresEquis | hno: you have a cache lying around ;)? | 21:03 |
jam | lifeless: morning. I'm trying to get reviews done, but otherwise I'd appreciate the discussion | 21:03 |
lifeless | jam: code or peers ? | 21:04 |
lifeless | peers we have until june 11 I think | 21:04 |
jam | peers | 21:05 |
lifeless | so tell me when you have a break | 21:06 |
jam | lifeless: did you make the peer requests, or was it done for you? | 21:06 |
lifeless | you make them yourself - on the left bottom there is 'add peer' | 21:06 |
lifeless | or something to that effect | 21:06 |
lifeless | we had to have the mgr review sent in monday, then there is a gap | 21:07 |
hno | TresEquis: Non-obvious how to make a cache to bzr with all the smartcalls around.. | 21:08 |
jam | lifeless: so what's up? | 21:36 |
lifeless | well | 21:36 |
lifeless | we seem to be sending 2-3 times the repo size on smart push of a full branch | 21:36 |
lifeless | 200MB to push bzr.dev to launchpad when its not stacked | 21:36 |
lifeless | I struggle to believe that its the encoding overhead | 21:36 |
lifeless | and, possibly related, we're not saturing the network at all effectively | 21:37 |
lifeless | I'm patch pilot this week, so not all that well placed to just nose down and drill into this. | 21:37 |
lifeless | s/uring/urating/ | 21:37 |
lifeless | I'm hoping that you or spiv will be between patches and able to look deeply into it | 21:38 |
mkanat | lifeless: I think a smart pull of a full branch is the same, as well. | 21:39 |
jam | lifeless: I'm 75% confident that it is the "split to organize 'properly'" code, and dependent on how the sort order turns up. If you want a quick and dirty test, just change the fetch order to 'unordered' and see if it doesn't drop the bytes transferred. | 21:39 |
lifeless | mkanat: yes, I'd expect that | 21:39 |
mkanat | lifeless: Oh, yeah, I suppose because of the indexes and all. | 21:39 |
jam | mkanat: no indices for smart pull, but push should transfer ~ the same as pull | 21:40 |
lifeless | mkanat: just because push and pull on the smart server are symmetrical, its just which end does the work | 21:40 |
lifeless | jam: so I can do quick tests | 21:41 |
lifeless | jam: what I want, is to know that someone else is going to run it down to ground ;) | 21:41 |
jam | lifeless: I've looked at this a few times, and mentioned it, and didn't get much traction anywhere... | 21:42 |
lifeless | jam: traction with people or code ? | 21:42 |
jam | people | 21:42 |
jam | If we really want minimal transfer then we need to compress on the server isde | 21:42 |
jam | side | 21:42 |
lifeless | ok | 21:42 |
jam | IMO | 21:42 |
lifeless | makes sense to me | 21:42 |
lifeless | I've been saying lp needs to scale with multiple front-end CPU boxes for a while, so it won't be a total suprise there ;) | 21:43 |
jam | I think we were avoiding it because of wanting to avoid load on a central location | 21:43 |
jam | though still a question as to whether push or pull is more common :) | 21:43 |
jam | most likely fetching from lp is the most common. | 21:43 |
lifeless | I don't recall any decision to avoid it | 21:43 |
jam | lifeless: we've certainly discussed wanting clients to do a bit more work than a server, since you usually have N clients and 1 server. | 21:44 |
lifeless | thats true | 21:44 |
lifeless | but, if wishes were horses ;) | 21:44 |
lifeless | are there any stats we should be gathering from the server to tell if this is the case ? | 21:45 |
jam | lifeless: the "split group" comments would be a likely point | 21:45 |
lifeless | ok | 21:45 |
jam | since it is taking existing groups, and splitting them into less efficient ones | 21:45 |
mkanat | jam: If there are no indices for smart pull, then what makes the pull so large? | 21:45 |
jam | the big thing that I've seen | 21:45 |
lifeless | well something I can follow through is getting some better logging and stats | 21:45 |
lifeless | mkanat: bugs ! :P | 21:45 |
mkanat | lifeless: Hahaha, okay. :-) | 21:46 |
jam | is when it cycles between 2 groups over and over | 21:46 |
jam | taking some bits from A, then B, then A, then B | 21:46 |
jam | those are definitely going to put a lot more data on the wire, as each split == fulltext | 21:46 |
lifeless | so, CHKRepo format is 'unordered' already | 21:47 |
lifeless | and StreamSource is groupcompress | 21:47 |
jam | bzrlib/repo_fmt/groupcompress_repo.py line 1025 | 21:48 |
jam | set that to 'unordered' | 21:48 |
lifeless | yeah | 21:48 |
lifeless | vila: thanks - Working tree "/home/robertc/source/baz/bzr-test-fixes/" has uncommitted changes (See bzr status). Uncommitted changes will not be pushed. - much better. | 21:49 |
lifeless | ok, finding revisions taking a bit | 21:51 |
jam | lifeless: so I just pushed on the local network, and saw 101.5MB to transfer a random bzr branch. | 21:52 |
jam | size on disk when finished was 36MB | 21:52 |
lifeless | 3x | 21:52 |
jam | well, size in 'upload' | 21:52 |
jam | no indices written yet | 21:52 |
lifeless | yes | 21:52 |
jam | still spinning on the 'commit_write_group' I think | 21:52 |
lifeless | yes, we need to address that too | 21:52 |
jam | lifeless: one other thing to 'address' is that I think the 'should this be rebuilt' heuristic is wrong for chk pages. As it checks to see if a group is 'full enough' and if not, it rebuilds it. But chk groups got better compression by logical grouping than trying to make the groups max size | 21:54 |
lifeless | ok | 21:54 |
jam | hmm... I just waited maybe 5-10 min after pushing, and now it is on Missing Keys. Is it going to be doing the check *yet again* ... | 21:55 |
lifeless | would that affect the A<->B<->A behaviour you mentin abve ? | 21:55 |
jam | lifeless: I don't think so | 21:55 |
jam | I'm guessing the A B A stuff is more because of inexact sorting | 21:55 |
jam | The sort is stable if you have the same set of revisions | 21:55 |
lifeless | ok, streaming @ 70KB | 21:55 |
jam | but I don't know how stable it is when new stuff is added. | 21:56 |
lifeless | I'm on adsl, so thats maybe 1/3 of my available b/w | 21:56 |
jam | lifeless: well, I'm testing on the home network, and certainly not saturated. | 22:00 |
jam | though I'm in the basement with the WAP 2 stories up | 22:00 |
jam | lifeless: sending from a freshly populated repo, which only has the content in question | 22:02 |
jam | and I see 52.9MB transferred | 22:02 |
jam | note that size on disk post indicies is 50MB | 22:02 |
jam | 12M in indices, 39M in packs | 22:02 |
jam | lifeless: in the latter case, I see 18: "creating new compressed block" lines in .bzr.log, in the former, it is closer to 3,300 | 22:05 |
jam | lifeless: case 3, push from unorganized repo, using 'unordered' texts, shows about 73.2MB transferred | 22:12 |
lifeless | I see many pages | 22:14 |
lifeless | not 3.3K worth | 22:14 |
lifeless | 12 pages | 22:14 |
lifeless | 25 per page - 300 | 22:14 |
lifeless | 1082.823 Transferred: 68957kB (63.8kB/s r:2368kB w:66588kB) | 22:15 |
lifeless | that was with unordered | 22:16 |
lifeless | so I image it would be considerably worse without | 22:16 |
lifeless | I can sftp upload a file at roughly the same speed | 22:17 |
lifeless | +- < 10% | 22:17 |
GaryvdM | vila: I've merge lp:~garyvdm/qbzr/annotate_find into lp:qbzr | 22:36 |
jam | lifeless: @64kB? | 22:38 |
jam | lifeless: so your unordered and my unordered seem on par | 22:38 |
guijemont | GaryvdM: hi, i have a qlog question | 22:38 |
lifeless | yeah, 68, 71, etc | 22:38 |
GaryvdM | Hi guijemont. | 22:38 |
lifeless | jam: no, unordered is ~= SFTP | 22:38 |
lifeless | using lftp to upload a single pack | 22:39 |
jam | lifeless: I mean that uploading via lftp you get 64kB/s | 22:39 |
guijemont | GaryvdM: when expanding a node, some other nodes (with links to the expanded stuff) are expanded | 22:39 |
lifeless | so, I'm getting wire speed with unordered | 22:39 |
lifeless | or at least, wire + latency to london | 22:39 |
jam | ... Transferred ... (63.8kB/s ...) | 22:39 |
guijemont | but there seems to be a limit on the quantity of branches that is expanded | 22:40 |
guijemont | but I don't get what that limit is nor how/where it is implemented | 22:40 |
lifeless | `../.bzr/repository/packs/2fc2a401beb2fd09d041842e3c27ebb7.pack' at 101785571 (76%) 61.4K/s eta:7m [Sending data] | 22:40 |
GaryvdM | guijemont: ah - let me have a look at the code. | 22:40 |
guijemont | what I was looking at is colapse_expand_rev(), but I'm not sure it's the right place | 22:41 |
GaryvdM | guijemont: That is the right place | 22:41 |
guijemont | esp. since it seems ot be called with visible=True, which results to a quite simple codepath | 22:41 |
GaryvdM | guijemont: It starts by looking at rev.twisty_branch_ids | 22:46 |
GaryvdM | guijemont: rev.twisty_branch_ids gets updated each time compute_graph_lines runs | 22:47 |
guijemont | hmm, and does that explain how some parents get expanded, and some other not (apparently, with lines drawn with dots) | 22:52 |
guijemont | ? | 22:52 |
GaryvdM | guijemont: It remebers what caused a branch to expand. | 22:54 |
GaryvdM | guijemont: e.g.: http://pastebin.org/276155 | 22:54 |
guijemont | oh, ok, and it doesn't go more than a given number of recursions into that? | 22:55 |
GaryvdM | guijemont: if you expand 4, and then 3 and then collapse 3, branch line 1.1 will stay open, because 4 expanded it | 22:56 |
guijemont | yeah, i noticed that | 22:56 |
GaryvdM | guijemont: but if you expand 3 and then 2.1.1, then collapse 3, branch line 1.1 will collapse | 22:57 |
GaryvdM | guijemont: This is what self.branch_lines[branch_id].expanded_by is for | 22:57 |
GaryvdM | guijemont: If collapsing, the we 'recurse' into branch_line.merges | 22:59 |
guijemont | ok | 22:59 |
guijemont | not exactly my issue though | 22:59 |
guijemont | hold on | 22:59 |
guijemont | making a screenshot | 23:00 |
GaryvdM | guijemont: processed_branch_ids is to track what we have looked at so that we don't do a branch more than once, which prevents a infinite 'recursion'/loop. | 23:01 |
guijemont | GaryvdM: http://emont.org/dots.png | 23:02 |
guijemont | there, some lines are drawn with dots | 23:02 |
guijemont | and some of them go to the "merge level 0 ancestor" of the bzr parent | 23:02 |
guijemont | without expanding happening | 23:02 |
GaryvdM | guijemont: hmm... | 23:03 |
GaryvdM | guijemont: the dots indicate that the line is going to a non intimidate ancestor. | 23:04 |
GaryvdM | guijemont: what happens if you click on say 2972.4.2, and then click on the other parent in the lower plane? | 23:05 |
guijemont | hold on, i closed the window... | 23:06 |
guijemont | and what's a "non intimidate ancestor"? | 23:07 |
GaryvdM | guijemont: The line layout algorithm I use dose not suit the mysql dags well. :-( | 23:07 |
guijemont | well, I guess mysql is a good test | 23:08 |
guijemont | since I have that bug with it with my change to viz | 23:08 |
guijemont | because my method to expand to the parents | 23:09 |
guijemont | is to do it one level at a time | 23:09 |
guijemont | in a callback that gets called when a row is expanded | 23:09 |
guijemont | so, for each new row that I expand, the callback gets called again | 23:10 |
guijemont | but in the case of mysql, the depdth at which you go with that is too big | 23:10 |
guijemont | and I end up with a maximum recursion exception | 23:10 |
guijemont | so i wanna try to do that iteratively instead | 23:11 |
GaryvdM | guijemont: e.g.: http://pastebin.org/276155 - if branch line (bl) 1.1 is visible, and bl 2.1 was hidden, we would show a dotted line from 3 to 1.1.1. 1.1.1 is not a parent of 3, but rather a distant ancestor... | 23:11 |
GaryvdM | = "non intimidate ancestor" | 23:11 |
guijemont | GaryvdM: do you know of a document that explains that kind of vocabulary? | 23:12 |
fullermd | It's not polite to intimidate your ancestors. | 23:15 |
GaryvdM | guijemont: no sorry, these are things that I've kind of made up. I'm sure there are definition of these things in formal graph theory, which I have unfortunately not studied. | 23:15 |
guijemont | ok | 23:16 |
fullermd | ITYM "immediate"... | 23:16 |
GaryvdM | fullermd: yes - thanks | 23:16 |
guijemont | ahh | 23:17 |
guijemont | oh | 23:17 |
guijemont | ok | 23:17 |
guijemont | might be clearer with "immediate" | 23:17 |
GaryvdM | jam is seems very knowledgeable about graph theory. Maybe he can give us the correct terms. | 23:17 |
* fullermd thinks there need to be more terms like "weird uncle". | 23:18 | |
GaryvdM | guijemont: In the code, I refer to non immediate ancestor as direct = False | 23:19 |
fullermd | How about 'indirect'. | 23:20 |
GaryvdM | direct = False / indirect = True | 23:20 |
fullermd | I mean as a term. | 23:20 |
guijemont | I'm sure I *knew* all this graph theory vocabulary well | 23:21 |
fullermd | Makes sense as the opposite of direct. | 23:21 |
guijemont | though I knew it in french... | 23:21 |
GaryvdM | guijemont: It also does not help that I'm terrible at spelling :-( | 23:22 |
guijemont | hu | 23:23 |
guijemont | btw, I think colapse_expand_rev() should be called collapse_expand_rev() ;) | 23:24 |
lifeless | yes | 23:24 |
GaryvdM | yes - that is one of my spelling mistakes.... | 23:24 |
guijemont | anyway, I think I get how the dotted lines work now | 23:24 |
guijemont | at least the idea, which is what interests me now | 23:25 |
guijemont | GaryvdM: thanks! | 23:25 |
GaryvdM | guijemont: Pleasure. | 23:25 |
GaryvdM | Spelling of 'collapse' fixed. :-) | 23:34 |
=== sebi_` is now known as sebi` | ||
guijemont | cool, so I did something useful today, i reported a bug that got fixed :) | 23:40 |
guijemont | now I feel productive | 23:41 |
guijemont | I can go to bed | 23:41 |
lifeless | mwhudson: hi | 23:43 |
mwhudson | lifeless: hello | 23:43 |
lifeless | mwhudson: I'd like to get stats on bzr smart server stuff on production | 23:43 |
lifeless | I'm sure you've considered this. | 23:43 |
lifeless | care to tell me the shortest path ? :) | 23:43 |
mwhudson | lifeless: 'stats' ? | 23:47 |
lifeless | how many split groups per push/pull | 23:47 |
lifeless | vs unsplit groups | 23:47 |
lifeless | total bytes sents | 23:47 |
lifeless | possibly drilling in further | 23:48 |
mwhudson | lifeless: i guess something like: | 23:48 |
* mwhudson thinks a bit more | 23:48 | |
lifeless | like | 23:48 |
lifeless | I'm not scared if you say 'change bzr, change bzr-lp-serve, change oops, and change this odd crnoscript | 23:48 |
mwhudson | lifeless: i guess something like: arrange for the bzr serve processes to log somewhere more obvious than ~/.bzr.log | 23:48 |
mwhudson | lifeless: then make sure that the bzr serve processes log the information you want to see (possibly with a -D style flag i guess? maybe this already exists) | 23:49 |
mwhudson | lifeless: arrange with sysadmins to copy log files somewhere you can see them | 23:49 |
mwhudson | lifeless: write scripts to parse said log files | 23:50 |
mwhudson | lifeless: none of these steps seems particularly hard | 23:50 |
lifeless | ok | 23:50 |
lifeless | whats unobvious about ~/.bzr.log ? | 23:50 |
mwhudson | i got a bit frustrated at one point because i wanted to have the bzr serve process log to a file like bzr-serve-$PID.log but by the time you know the pid it's too late to set $BZR_LOG_FILE (or whatever that's called) | 23:51 |
mwhudson | lifeless: well, perhaps 'obvious' was the wrong word | 23:51 |
lifeless | ok | 23:51 |
lifeless | what would a better word be? | 23:51 |
mwhudson | lifeless: it has the output of all the bzr serve processes jumbled up in it, it's not synced anywhere, and it gets rotated by bzr's policy not the sysadmin's | 23:52 |
lifeless | ok, the first point is fine (we put pids in), the sync issue we can fix, the rotation however - we should fix that in bzr. | 23:52 |
lifeless | log_rotation = None in ~/.bazaar.conf ? | 23:53 |
lifeless | .bazaar/bazaar.conf, I mean | 23:53 |
mwhudson | i guess that works | 23:53 |
lifeless | We can work up to other things | 23:53 |
lifeless | alternatively | 23:59 |
lifeless | perhaps | 23:59 |
lifeless | log_file = ~/.bazaar/logs/$time-$pid.log | 23:59 |
lifeless | in bazaar.conf | 23:59 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!